2026-02-24T18:14:03.832 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-24T18:14:03.847 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T18:14:03.927 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} email: null first_in_suite: false flavor: default job_id: '68651' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 160 osd op thread timeout: 150 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 240 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request - OSD bench result sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 stop-daemons-timeout: 1200 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 5907 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiOv4wxAd5q5Cfw+X+/1OArumqTctOp8Jn+J/erK9HsZligMjUkVxX0bN61dka3O7FWelZ7pyRxhDWc1V3KNsQ= trial032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL8xNQnp/KmEH/gR6flU2UiYBKNnUKXkFMcfIQvOqLhf10RfYLsG2h5Hn3sZHPqT1IT849eIPOew1mc5hPHjRwc= trial110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCaNdVXemvq4+d7G9yuKi7aoK94ARoZOpiql99rbT7YanP/B/wKy6n0UffTwiZRa5B8E22IZTPPYQE6yUZa9dXI= trial112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/B8xTpilT/W17zf15Y8jwxEsmW/8o+yZwlILDFqq3pw7dT+fodE953+WOhU6hPpeQtL5kw5gLOeY0p40lQ4Qw= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - '-- Don''t run this test on tiny clusters if py_len(yaml.roles) < 4 then reject() end ' teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-24_16:22:43 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T18:14:03.927 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-24T18:14:03.928 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-24T18:14:03.928 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-24T18:14:03.929 INFO:teuthology.task.internal:Checking packages... 2026-02-24T18:14:03.929 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-24T18:14:03.929 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-24T18:14:03.929 INFO:teuthology.packaging:ref: None 2026-02-24T18:14:03.929 INFO:teuthology.packaging:tag: None 2026-02-24T18:14:03.929 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-24T18:14:03.929 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:14:03.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-24T18:14:04.047 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689.gf90f4eae 2026-02-24T18:14:04.048 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-24T18:14:04.063 INFO:teuthology.task.internal:no buildpackages task found 2026-02-24T18:14:04.063 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T18:14:04.077 INFO:teuthology.task.internal:Saving configuration 2026-02-24T18:14:04.083 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T18:14:04.095 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T18:14:04.152 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial018.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 18:10:22.694222', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiOv4wxAd5q5Cfw+X+/1OArumqTctOp8Jn+J/erK9HsZligMjUkVxX0bN61dka3O7FWelZ7pyRxhDWc1V3KNsQ='} 2026-02-24T18:14:04.210 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial032.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 18:10:22.692847', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL8xNQnp/KmEH/gR6flU2UiYBKNnUKXkFMcfIQvOqLhf10RfYLsG2h5Hn3sZHPqT1IT849eIPOew1mc5hPHjRwc='} 2026-02-24T18:14:04.272 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial110.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 18:10:22.693847', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCaNdVXemvq4+d7G9yuKi7aoK94ARoZOpiql99rbT7YanP/B/wKy6n0UffTwiZRa5B8E22IZTPPYQE6yUZa9dXI='} 2026-02-24T18:14:04.326 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial112.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 18:10:22.693460', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/B8xTpilT/W17zf15Y8jwxEsmW/8o+yZwlILDFqq3pw7dT+fodE953+WOhU6hPpeQtL5kw5gLOeY0p40lQ4Qw='} 2026-02-24T18:14:04.326 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T18:14:04.338 INFO:teuthology.task.internal:roles: ubuntu@trial018.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2026-02-24T18:14:04.338 INFO:teuthology.task.internal:roles: ubuntu@trial032.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2026-02-24T18:14:04.338 INFO:teuthology.task.internal:roles: ubuntu@trial110.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2026-02-24T18:14:04.338 INFO:teuthology.task.internal:roles: ubuntu@trial112.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2026-02-24T18:14:04.338 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T18:14:04.596 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7362e3b960c0>, signals=[15]) 2026-02-24T18:14:04.596 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T18:14:04.608 INFO:teuthology.task.internal:Opening connections... 2026-02-24T18:14:04.608 DEBUG:teuthology.task.internal:connecting to ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:14:04.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:14:04.670 DEBUG:teuthology.task.internal:connecting to ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:14:04.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:14:04.737 DEBUG:teuthology.task.internal:connecting to ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:14:04.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:14:04.797 DEBUG:teuthology.task.internal:connecting to ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:14:04.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:14:04.857 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T18:14:04.870 DEBUG:teuthology.orchestra.run.trial018:> uname -m 2026-02-24T18:14:04.885 INFO:teuthology.orchestra.run.trial018.stdout:x86_64 2026-02-24T18:14:04.885 DEBUG:teuthology.orchestra.run.trial018:> cat /etc/os-release 2026-02-24T18:14:04.940 INFO:teuthology.orchestra.run.trial018.stdout:NAME="CentOS Stream" 2026-02-24T18:14:04.940 INFO:teuthology.orchestra.run.trial018.stdout:VERSION="9" 2026-02-24T18:14:04.940 INFO:teuthology.orchestra.run.trial018.stdout:ID="centos" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:ID_LIKE="rhel fedora" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:VERSION_ID="9" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:PLATFORM_ID="platform:el9" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:ANSI_COLOR="0;31" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:LOGO="fedora-logo-icon" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:HOME_URL="https://centos.org/" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T18:14:04.941 INFO:teuthology.orchestra.run.trial018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T18:14:04.942 INFO:teuthology.lock.ops:Updating trial018.front.sepia.ceph.com on lock server 2026-02-24T18:14:05.017 DEBUG:teuthology.orchestra.run.trial032:> uname -m 2026-02-24T18:14:05.033 INFO:teuthology.orchestra.run.trial032.stdout:x86_64 2026-02-24T18:14:05.033 DEBUG:teuthology.orchestra.run.trial032:> cat /etc/os-release 2026-02-24T18:14:05.088 INFO:teuthology.orchestra.run.trial032.stdout:NAME="CentOS Stream" 2026-02-24T18:14:05.088 INFO:teuthology.orchestra.run.trial032.stdout:VERSION="9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:ID="centos" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:ID_LIKE="rhel fedora" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:VERSION_ID="9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:PLATFORM_ID="platform:el9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:ANSI_COLOR="0;31" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:LOGO="fedora-logo-icon" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:HOME_URL="https://centos.org/" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T18:14:05.089 INFO:teuthology.orchestra.run.trial032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T18:14:05.089 INFO:teuthology.lock.ops:Updating trial032.front.sepia.ceph.com on lock server 2026-02-24T18:14:05.162 DEBUG:teuthology.orchestra.run.trial110:> uname -m 2026-02-24T18:14:05.177 INFO:teuthology.orchestra.run.trial110.stdout:x86_64 2026-02-24T18:14:05.178 DEBUG:teuthology.orchestra.run.trial110:> cat /etc/os-release 2026-02-24T18:14:05.233 INFO:teuthology.orchestra.run.trial110.stdout:NAME="CentOS Stream" 2026-02-24T18:14:05.233 INFO:teuthology.orchestra.run.trial110.stdout:VERSION="9" 2026-02-24T18:14:05.233 INFO:teuthology.orchestra.run.trial110.stdout:ID="centos" 2026-02-24T18:14:05.233 INFO:teuthology.orchestra.run.trial110.stdout:ID_LIKE="rhel fedora" 2026-02-24T18:14:05.233 INFO:teuthology.orchestra.run.trial110.stdout:VERSION_ID="9" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:PLATFORM_ID="platform:el9" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:ANSI_COLOR="0;31" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:LOGO="fedora-logo-icon" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:HOME_URL="https://centos.org/" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T18:14:05.234 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T18:14:05.234 INFO:teuthology.lock.ops:Updating trial110.front.sepia.ceph.com on lock server 2026-02-24T18:14:05.305 DEBUG:teuthology.orchestra.run.trial112:> uname -m 2026-02-24T18:14:05.320 INFO:teuthology.orchestra.run.trial112.stdout:x86_64 2026-02-24T18:14:05.320 DEBUG:teuthology.orchestra.run.trial112:> cat /etc/os-release 2026-02-24T18:14:05.375 INFO:teuthology.orchestra.run.trial112.stdout:NAME="CentOS Stream" 2026-02-24T18:14:05.375 INFO:teuthology.orchestra.run.trial112.stdout:VERSION="9" 2026-02-24T18:14:05.375 INFO:teuthology.orchestra.run.trial112.stdout:ID="centos" 2026-02-24T18:14:05.375 INFO:teuthology.orchestra.run.trial112.stdout:ID_LIKE="rhel fedora" 2026-02-24T18:14:05.375 INFO:teuthology.orchestra.run.trial112.stdout:VERSION_ID="9" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:PLATFORM_ID="platform:el9" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:ANSI_COLOR="0;31" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:LOGO="fedora-logo-icon" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:HOME_URL="https://centos.org/" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T18:14:05.376 INFO:teuthology.orchestra.run.trial112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T18:14:05.376 INFO:teuthology.lock.ops:Updating trial112.front.sepia.ceph.com on lock server 2026-02-24T18:14:05.434 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T18:14:05.447 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T18:14:05.468 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T18:14:05.468 DEBUG:teuthology.orchestra.run.trial018:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:14:05.469 DEBUG:teuthology.orchestra.run.trial032:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:14:05.471 DEBUG:teuthology.orchestra.run.trial110:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:14:05.473 DEBUG:teuthology.orchestra.run.trial112:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:14:05.488 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T18:14:05.502 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T18:14:05.503 DEBUG:teuthology.orchestra.run.trial018:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:14:05.529 DEBUG:teuthology.orchestra.run.trial032:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:14:05.534 DEBUG:teuthology.orchestra.run.trial110:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:14:05.541 DEBUG:teuthology.orchestra.run.trial112:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:14:05.565 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T18:14:05.824 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T18:14:05.844 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T18:14:05.844 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.c': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T18:14:05.845 DEBUG:teuthology.orchestra.run.trial018:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:14:05.845 DEBUG:teuthology.orchestra.run.trial032:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:14:05.845 DEBUG:teuthology.orchestra.run.trial110:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:14:05.845 DEBUG:teuthology.orchestra.run.trial112:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:14:05.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:14:05.860 DEBUG:teuthology.orchestra.run.trial110:> uname -r 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run.trial018:> uname -r 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run.trial032:> uname -r 2026-02-24T18:14:05.861 DEBUG:teuthology.orchestra.run.trial112:> uname -r 2026-02-24T18:14:05.915 INFO:teuthology.orchestra.run.trial110.stdout:5.14.0-681.el9.x86_64 2026-02-24T18:14:05.915 INFO:teuthology.task.kernel:Running kernel on trial110: 5.14.0-681.el9.x86_64 2026-02-24T18:14:05.915 DEBUG:teuthology.orchestra.run.trial110:> sudo yum install -y kernel 2026-02-24T18:14:05.917 INFO:teuthology.orchestra.run.trial112.stdout:5.14.0-681.el9.x86_64 2026-02-24T18:14:05.917 INFO:teuthology.task.kernel:Running kernel on trial112: 5.14.0-681.el9.x86_64 2026-02-24T18:14:05.917 DEBUG:teuthology.orchestra.run.trial112:> sudo yum install -y kernel 2026-02-24T18:14:05.918 INFO:teuthology.orchestra.run.trial018.stdout:5.14.0-681.el9.x86_64 2026-02-24T18:14:05.918 INFO:teuthology.orchestra.run.trial032.stdout:5.14.0-681.el9.x86_64 2026-02-24T18:14:05.918 INFO:teuthology.task.kernel:Running kernel on trial018: 5.14.0-681.el9.x86_64 2026-02-24T18:14:05.918 DEBUG:teuthology.orchestra.run.trial018:> sudo yum install -y kernel 2026-02-24T18:14:05.918 INFO:teuthology.task.kernel:Running kernel on trial032: 5.14.0-681.el9.x86_64 2026-02-24T18:14:05.918 DEBUG:teuthology.orchestra.run.trial032:> sudo yum install -y kernel 2026-02-24T18:14:06.624 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - BaseOS 27 MB/s | 8.9 MB 00:00 2026-02-24T18:14:06.973 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-24T18:14:06.987 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-24T18:14:07.123 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-24T18:14:09.685 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-24T18:14:10.310 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-24T18:14:11.177 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 27 MB 00:03 2026-02-24T18:14:12.605 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - CRB 15 MB/s | 8.0 MB 00:00 2026-02-24T18:14:14.655 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 20 kB 00:01 2026-02-24T18:14:14.809 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 8.0 MB 00:01 2026-02-24T18:14:14.973 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 8.0 MB 00:02 2026-02-24T18:14:15.701 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - Extras packages 155 kB/s | 20 kB 00:00 2026-02-24T18:14:15.940 INFO:teuthology.orchestra.run.trial110.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-24T18:14:16.078 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 20 kB 00:00 2026-02-24T18:14:16.928 INFO:teuthology.orchestra.run.trial112.stdout:Extra Packages for Enterprise Linux 26 MB/s | 20 MB 00:00 2026-02-24T18:14:17.269 INFO:teuthology.orchestra.run.trial018.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2026-02-24T18:14:19.144 INFO:teuthology.orchestra.run.trial110.stdout:lab-extras 987 kB/s | 50 kB 00:00 2026-02-24T18:14:19.351 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 27 MB 00:11 2026-02-24T18:14:20.129 INFO:teuthology.orchestra.run.trial112.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-24T18:14:20.155 INFO:teuthology.orchestra.run.trial110.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T18:14:20.156 INFO:teuthology.orchestra.run.trial110.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T18:14:20.176 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout: Package Architecture Version Repository Size 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T18:14:20.177 INFO:teuthology.orchestra.run.trial110.stdout:Installing dependencies: 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:Install 4 Packages 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 89 M 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 135 M 2026-02-24T18:14:20.178 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:14:20.312 INFO:teuthology.orchestra.run.trial110.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 12 MB/s | 901 kB 00:00 2026-02-24T18:14:20.457 INFO:teuthology.orchestra.run.trial018.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-24T18:14:20.661 INFO:teuthology.orchestra.run.trial110.stdout:(2/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 94 MB/s | 40 MB 00:00 2026-02-24T18:14:21.106 INFO:teuthology.orchestra.run.trial110.stdout:(3/4): kernel-core-5.14.0-686.el9.x86_64.rpm 20 MB/s | 17 MB 00:00 2026-02-24T18:14:21.142 INFO:teuthology.orchestra.run.trial112.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T18:14:21.142 INFO:teuthology.orchestra.run.trial112.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T18:14:21.162 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout: Package Architecture Version Repository Size 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout:Installing dependencies: 2026-02-24T18:14:21.164 INFO:teuthology.orchestra.run.trial112.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:Install 4 Packages 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 89 M 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 135 M 2026-02-24T18:14:21.165 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:14:21.306 INFO:teuthology.orchestra.run.trial110.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 31 MB/s | 31 MB 00:00 2026-02-24T18:14:21.306 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:14:21.306 INFO:teuthology.orchestra.run.trial110.stdout:Total 79 MB/s | 89 MB 00:01 2026-02-24T18:14:21.475 INFO:teuthology.orchestra.run.trial018.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T18:14:21.475 INFO:teuthology.orchestra.run.trial018.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T18:14:21.495 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:14:21.496 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:14:21.496 INFO:teuthology.orchestra.run.trial018.stdout: Package Architecture Version Repository Size 2026-02-24T18:14:21.496 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:14:21.496 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout:Installing dependencies: 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout:Install 4 Packages 2026-02-24T18:14:21.497 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:21.498 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 89 M 2026-02-24T18:14:21.498 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 135 M 2026-02-24T18:14:21.498 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:14:21.498 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:14:21.699 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:14:21.699 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:14:21.788 INFO:teuthology.orchestra.run.trial018.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 3.8 MB/s | 901 kB 00:00 2026-02-24T18:14:21.946 INFO:teuthology.orchestra.run.trial018.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 44 MB/s | 17 MB 00:00 2026-02-24T18:14:22.082 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:14:22.082 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:14:22.119 INFO:teuthology.orchestra.run.trial018.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 94 MB/s | 31 MB 00:00 2026-02-24T18:14:22.141 INFO:teuthology.orchestra.run.trial112.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 981 kB/s | 901 kB 00:00 2026-02-24T18:14:22.200 INFO:teuthology.orchestra.run.trial018.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 62 MB/s | 40 MB 00:00 2026-02-24T18:14:22.200 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:14:22.200 INFO:teuthology.orchestra.run.trial018.stdout:Total 126 MB/s | 89 MB 00:00 2026-02-24T18:14:22.393 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:14:22.597 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:14:22.598 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:14:22.608 INFO:teuthology.orchestra.run.trial112.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 12 MB/s | 17 MB 00:01 2026-02-24T18:14:22.794 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 8.0 MB 00:01 2026-02-24T18:14:22.984 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:14:22.984 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:14:23.110 INFO:teuthology.orchestra.run.trial112.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-24T18:14:23.194 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:14:23.483 INFO:teuthology.orchestra.run.trial110.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:23.505 INFO:teuthology.orchestra.run.trial110.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:23.647 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - Extras packages 174 kB/s | 20 kB 00:00 2026-02-24T18:14:23.827 INFO:teuthology.orchestra.run.trial112.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 18 MB/s | 31 MB 00:01 2026-02-24T18:14:23.828 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:14:23.828 INFO:teuthology.orchestra.run.trial112.stdout:Total 33 MB/s | 89 MB 00:02 2026-02-24T18:14:24.021 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:14:24.091 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:14:24.125 INFO:teuthology.orchestra.run.trial032.stdout:Extra Packages for Enterprise Linux 50 MB/s | 20 MB 00:00 2026-02-24T18:14:24.223 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:14:24.223 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:14:24.380 INFO:teuthology.orchestra.run.trial018.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:24.402 INFO:teuthology.orchestra.run.trial018.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:24.605 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:14:24.605 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:14:25.712 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:14:26.000 INFO:teuthology.orchestra.run.trial112.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:26.022 INFO:teuthology.orchestra.run.trial112.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:26.088 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:26.121 INFO:teuthology.orchestra.run.trial110.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:26.971 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:27.004 INFO:teuthology.orchestra.run.trial018.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:27.317 INFO:teuthology.orchestra.run.trial032.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-24T18:14:28.342 INFO:teuthology.orchestra.run.trial032.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T18:14:28.343 INFO:teuthology.orchestra.run.trial032.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T18:14:28.362 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: Package Architecture Version Repository Size 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout:Installing dependencies: 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:28.363 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout:Install 4 Packages 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 89 M 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 135 M 2026-02-24T18:14:28.364 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:14:28.582 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:28.615 INFO:teuthology.orchestra.run.trial112.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:28.623 INFO:teuthology.orchestra.run.trial032.stdout:(1/4): kernel-core-5.14.0-686.el9.x86_64.rpm 86 MB/s | 17 MB 00:00 2026-02-24T18:14:28.817 INFO:teuthology.orchestra.run.trial032.stdout:(2/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 101 MB/s | 40 MB 00:00 2026-02-24T18:14:29.165 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:29.171 INFO:teuthology.orchestra.run.trial110.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:29.639 INFO:teuthology.orchestra.run.trial032.stdout:(3/4): kernel-5.14.0-686.el9.x86_64.rpm 741 kB/s | 901 kB 00:01 2026-02-24T18:14:29.938 INFO:teuthology.orchestra.run.trial032.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 24 MB/s | 31 MB 00:01 2026-02-24T18:14:29.940 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:14:29.940 INFO:teuthology.orchestra.run.trial032.stdout:Total 56 MB/s | 89 MB 00:01 2026-02-24T18:14:30.041 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:30.047 INFO:teuthology.orchestra.run.trial018.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:30.131 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:14:30.339 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:14:30.339 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:14:30.723 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:14:30.723 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:14:31.651 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:31.657 INFO:teuthology.orchestra.run.trial112.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:31.834 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:14:32.125 INFO:teuthology.orchestra.run.trial032.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:32.147 INFO:teuthology.orchestra.run.trial032.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:32.469 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:33.098 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:34.704 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:34.737 INFO:teuthology.orchestra.run.trial032.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:34.767 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:37.772 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:37.778 INFO:teuthology.orchestra.run.trial032.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:40.825 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:49.936 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:49.957 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.144 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.145 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:50.145 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:50.145 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:50.204 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:50.205 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:14:50.269 DEBUG:teuthology.orchestra.run.trial110:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:50.298 INFO:teuthology.orchestra.run.trial110.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.300 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.300 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:14:50.300 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:14:50.300 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2026-02-24T18:14:50.301 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial110.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:14:50.301 DEBUG:teuthology.orchestra.run.trial110:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:50.364 INFO:teuthology.orchestra.run.trial110.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.365 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.365 DEBUG:teuthology.orchestra.run.trial110:> sudo rpm -qi grub2-tools 2026-02-24T18:14:50.434 INFO:teuthology.orchestra.run.trial110.stdout:Name : grub2-tools 2026-02-24T18:14:50.434 INFO:teuthology.orchestra.run.trial110.stdout:Epoch : 1 2026-02-24T18:14:50.434 INFO:teuthology.orchestra.run.trial110.stdout:Version : 2.06 2026-02-24T18:14:50.434 INFO:teuthology.orchestra.run.trial110.stdout:Release : 120.el9 2026-02-24T18:14:50.434 INFO:teuthology.orchestra.run.trial110.stdout:Architecture: x86_64 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Group : Unspecified 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Size : 9108611 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:License : GPLv3+ 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Packager : builder@centos.org 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:Vendor : CentOS 2026-02-24T18:14:50.435 INFO:teuthology.orchestra.run.trial110.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:Summary : Support tools for GRUB. 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:Description : 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:hardware devices. 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:14:50.436 INFO:teuthology.orchestra.run.trial110.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T18:14:50.437 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T18:14:50.438 INFO:teuthology.task.kernel:Updating grub on trial110 to boot 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.438 DEBUG:teuthology.orchestra.run.trial110:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T18:14:50.500 INFO:teuthology.orchestra.run.trial110.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T18:14:50.501 DEBUG:teuthology.orchestra.run.trial110:> sudo command -v grubby && echo yes || echo no 2026-02-24T18:14:50.565 INFO:teuthology.orchestra.run.trial110.stdout:/sbin/grubby 2026-02-24T18:14:50.566 INFO:teuthology.orchestra.run.trial110.stdout:yes 2026-02-24T18:14:50.567 DEBUG:teuthology.orchestra.run.trial110:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T18:14:50.589 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.611 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.630 INFO:teuthology.orchestra.run.trial110.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T18:14:50.632 DEBUG:teuthology.orchestra.run.trial110:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.705 DEBUG:teuthology.orchestra.run.trial110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T18:14:50.798 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.799 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:50.799 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:50.799 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.856 INFO:teuthology.orchestra.run.trial018.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.857 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:50.857 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:14:50.913 INFO:teuthology.orchestra.run.trial110.stderr:Generating grub configuration file ... 2026-02-24T18:14:50.917 DEBUG:teuthology.orchestra.run.trial018:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:50.947 INFO:teuthology.orchestra.run.trial018.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.948 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.948 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T18:14:50.948 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:14:50.948 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:14:50.948 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T18:14:50.949 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial018.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:14:50.949 DEBUG:teuthology.orchestra.run.trial018:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:50.980 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T18:14:50.987 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T18:14:51.011 INFO:teuthology.orchestra.run.trial018.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:51.011 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:51.012 DEBUG:teuthology.orchestra.run.trial018:> sudo rpm -qi grub2-tools 2026-02-24T18:14:51.080 INFO:teuthology.orchestra.run.trial018.stdout:Name : grub2-tools 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Epoch : 1 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Version : 2.06 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Release : 120.el9 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Architecture: x86_64 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Group : Unspecified 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Size : 9108611 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:License : GPLv3+ 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Packager : builder@centos.org 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Vendor : CentOS 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T18:14:51.081 INFO:teuthology.orchestra.run.trial018.stdout:Summary : Support tools for GRUB. 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:Description : 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:hardware devices. 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:14:51.082 INFO:teuthology.orchestra.run.trial018.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T18:14:51.083 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T18:14:51.083 INFO:teuthology.task.kernel:Updating grub on trial018 to boot 5.14.0-686.el9.x86_64 2026-02-24T18:14:51.083 DEBUG:teuthology.orchestra.run.trial018:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T18:14:51.146 INFO:teuthology.orchestra.run.trial018.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T18:14:51.147 DEBUG:teuthology.orchestra.run.trial018:> sudo command -v grubby && echo yes || echo no 2026-02-24T18:14:51.211 INFO:teuthology.orchestra.run.trial018.stdout:/sbin/grubby 2026-02-24T18:14:51.212 INFO:teuthology.orchestra.run.trial018.stdout:yes 2026-02-24T18:14:51.213 DEBUG:teuthology.orchestra.run.trial018:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T18:14:51.240 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T18:14:51.246 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T18:14:51.276 INFO:teuthology.orchestra.run.trial018.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T18:14:51.276 DEBUG:teuthology.orchestra.run.trial018:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T18:14:51.282 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T18:14:51.288 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T18:14:51.319 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T18:14:51.325 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T18:14:51.348 DEBUG:teuthology.orchestra.run.trial018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T18:14:51.348 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T18:14:51.355 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T18:14:51.442 INFO:teuthology.orchestra.run.trial110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:14:51.450 INFO:teuthology.orchestra.run.trial110.stderr:done 2026-02-24T18:14:51.452 DEBUG:teuthology.orchestra.run.trial110:> test -d /sys/firmware/efi 2026-02-24T18:14:51.468 DEBUG:teuthology.orchestra.run.trial110:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T18:14:51.535 INFO:teuthology.orchestra.run.trial110.stdout:/boot/efi/EFI/BOOT 2026-02-24T18:14:51.535 INFO:teuthology.orchestra.run.trial110.stdout:/boot/efi/EFI/centos 2026-02-24T18:14:51.537 DEBUG:teuthology.orchestra.run.trial110:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T18:14:51.553 INFO:teuthology.orchestra.run.trial018.stderr:Generating grub configuration file ... 2026-02-24T18:14:51.603 INFO:teuthology.orchestra.run.trial110.stdout:no 2026-02-24T18:14:51.603 DEBUG:teuthology.orchestra.run.trial110:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T18:14:51.619 INFO:teuthology.orchestra.run.trial018.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T18:14:51.626 INFO:teuthology.orchestra.run.trial018.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T18:14:51.677 INFO:teuthology.orchestra.run.trial110.stdout:yes 2026-02-24T18:14:51.677 DEBUG:teuthology.orchestra.run.trial110:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T18:14:51.742 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial110 2026-02-24T18:14:51.742 DEBUG:teuthology.orchestra.run.trial110:> sudo bash -c ' 2026-02-24T18:14:51.742 DEBUG:teuthology.orchestra.run.trial110:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:14:51.742 DEBUG:teuthology.orchestra.run.trial110:> ' 2026-02-24T18:14:51.877 INFO:teuthology.orchestra.run.trial018.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T18:14:51.883 INFO:teuthology.orchestra.run.trial018.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T18:14:51.917 INFO:teuthology.orchestra.run.trial018.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T18:14:51.923 INFO:teuthology.orchestra.run.trial018.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T18:14:51.955 INFO:teuthology.orchestra.run.trial018.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T18:14:51.961 INFO:teuthology.orchestra.run.trial018.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T18:14:51.984 INFO:teuthology.orchestra.run.trial018.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T18:14:51.990 INFO:teuthology.orchestra.run.trial018.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T18:14:52.077 INFO:teuthology.orchestra.run.trial018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:14:52.085 INFO:teuthology.orchestra.run.trial018.stderr:done 2026-02-24T18:14:52.087 DEBUG:teuthology.orchestra.run.trial018:> test -d /sys/firmware/efi 2026-02-24T18:14:52.104 DEBUG:teuthology.orchestra.run.trial018:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T18:14:52.170 INFO:teuthology.orchestra.run.trial018.stdout:/boot/efi/EFI/BOOT 2026-02-24T18:14:52.170 INFO:teuthology.orchestra.run.trial018.stdout:/boot/efi/EFI/centos 2026-02-24T18:14:52.172 DEBUG:teuthology.orchestra.run.trial018:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T18:14:52.237 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:52.237 INFO:teuthology.orchestra.run.trial018.stdout:no 2026-02-24T18:14:52.238 DEBUG:teuthology.orchestra.run.trial018:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T18:14:52.258 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:52.301 INFO:teuthology.orchestra.run.trial018.stdout:yes 2026-02-24T18:14:52.302 DEBUG:teuthology.orchestra.run.trial018:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T18:14:52.364 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial018 2026-02-24T18:14:52.365 DEBUG:teuthology.orchestra.run.trial018:> sudo bash -c ' 2026-02-24T18:14:52.365 DEBUG:teuthology.orchestra.run.trial018:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:14:52.365 DEBUG:teuthology.orchestra.run.trial018:> ' 2026-02-24T18:14:52.444 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:52.445 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:52.445 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:52.445 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:52.501 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:52.502 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:14:52.563 DEBUG:teuthology.orchestra.run.trial112:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:52.591 INFO:teuthology.orchestra.run.trial112.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.592 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:52.592 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T18:14:52.592 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:14:52.592 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:14:52.592 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-24T18:14:52.592 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial112.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:14:52.592 DEBUG:teuthology.orchestra.run.trial112:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:52.660 INFO:teuthology.orchestra.run.trial112.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:52.661 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:52.661 DEBUG:teuthology.orchestra.run.trial112:> sudo rpm -qi grub2-tools 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Name : grub2-tools 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Epoch : 1 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Version : 2.06 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Release : 120.el9 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Architecture: x86_64 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Group : Unspecified 2026-02-24T18:14:52.730 INFO:teuthology.orchestra.run.trial112.stdout:Size : 9108611 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:License : GPLv3+ 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Packager : builder@centos.org 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Vendor : CentOS 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Summary : Support tools for GRUB. 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:Description : 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:hardware devices. 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:14:52.731 INFO:teuthology.orchestra.run.trial112.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T18:14:52.732 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T18:14:52.732 INFO:teuthology.task.kernel:Updating grub on trial112 to boot 5.14.0-686.el9.x86_64 2026-02-24T18:14:52.732 DEBUG:teuthology.orchestra.run.trial112:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T18:14:52.796 INFO:teuthology.orchestra.run.trial112.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T18:14:52.797 DEBUG:teuthology.orchestra.run.trial112:> sudo command -v grubby && echo yes || echo no 2026-02-24T18:14:52.862 INFO:teuthology.orchestra.run.trial112.stdout:/sbin/grubby 2026-02-24T18:14:52.863 INFO:teuthology.orchestra.run.trial112.stdout:yes 2026-02-24T18:14:52.863 DEBUG:teuthology.orchestra.run.trial112:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T18:14:52.927 INFO:teuthology.orchestra.run.trial112.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T18:14:52.928 DEBUG:teuthology.orchestra.run.trial112:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T18:14:53.000 DEBUG:teuthology.orchestra.run.trial112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T18:14:53.204 INFO:teuthology.orchestra.run.trial112.stderr:Generating grub configuration file ... 2026-02-24T18:14:53.271 INFO:teuthology.orchestra.run.trial112.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T18:14:53.277 INFO:teuthology.orchestra.run.trial112.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T18:14:53.525 INFO:teuthology.orchestra.run.trial112.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T18:14:53.532 INFO:teuthology.orchestra.run.trial112.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T18:14:53.566 INFO:teuthology.orchestra.run.trial112.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T18:14:53.573 INFO:teuthology.orchestra.run.trial112.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T18:14:53.604 INFO:teuthology.orchestra.run.trial112.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T18:14:53.610 INFO:teuthology.orchestra.run.trial112.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T18:14:53.633 INFO:teuthology.orchestra.run.trial112.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T18:14:53.639 INFO:teuthology.orchestra.run.trial112.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T18:14:53.725 INFO:teuthology.orchestra.run.trial112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:14:53.734 INFO:teuthology.orchestra.run.trial112.stderr:done 2026-02-24T18:14:53.736 DEBUG:teuthology.orchestra.run.trial112:> test -d /sys/firmware/efi 2026-02-24T18:14:53.753 DEBUG:teuthology.orchestra.run.trial112:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T18:14:53.818 INFO:teuthology.orchestra.run.trial112.stdout:/boot/efi/EFI/BOOT 2026-02-24T18:14:53.818 INFO:teuthology.orchestra.run.trial112.stdout:/boot/efi/EFI/centos 2026-02-24T18:14:53.820 DEBUG:teuthology.orchestra.run.trial112:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T18:14:53.884 INFO:teuthology.orchestra.run.trial112.stdout:no 2026-02-24T18:14:53.885 DEBUG:teuthology.orchestra.run.trial112:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T18:14:53.948 INFO:teuthology.orchestra.run.trial112.stdout:yes 2026-02-24T18:14:53.949 DEBUG:teuthology.orchestra.run.trial112:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T18:14:54.014 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial112 2026-02-24T18:14:54.014 DEBUG:teuthology.orchestra.run.trial112:> sudo bash -c ' 2026-02-24T18:14:54.015 DEBUG:teuthology.orchestra.run.trial112:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:14:54.015 DEBUG:teuthology.orchestra.run.trial112:> ' 2026-02-24T18:14:58.396 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:58.419 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:58.608 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:58.609 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:14:58.609 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:14:58.609 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:58.666 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:14:58.728 DEBUG:teuthology.orchestra.run.trial032:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:58.757 INFO:teuthology.orchestra.run.trial032.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.758 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:58.759 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T18:14:58.759 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:14:58.759 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:14:58.759 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-24T18:14:58.759 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial032.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:14:58.759 DEBUG:teuthology.orchestra.run.trial032:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:14:58.824 INFO:teuthology.orchestra.run.trial032.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:14:58.824 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:14:58.825 DEBUG:teuthology.orchestra.run.trial032:> sudo rpm -qi grub2-tools 2026-02-24T18:14:58.893 INFO:teuthology.orchestra.run.trial032.stdout:Name : grub2-tools 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Epoch : 1 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Version : 2.06 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Release : 120.el9 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Architecture: x86_64 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Group : Unspecified 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:Size : 9108611 2026-02-24T18:14:58.894 INFO:teuthology.orchestra.run.trial032.stdout:License : GPLv3+ 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Packager : builder@centos.org 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Vendor : CentOS 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Summary : Support tools for GRUB. 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:Description : 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T18:14:58.895 INFO:teuthology.orchestra.run.trial032.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T18:14:58.896 INFO:teuthology.orchestra.run.trial032.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T18:14:58.896 INFO:teuthology.orchestra.run.trial032.stdout:hardware devices. 2026-02-24T18:14:58.896 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:14:58.896 INFO:teuthology.orchestra.run.trial032.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T18:14:58.897 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T18:14:58.897 INFO:teuthology.task.kernel:Updating grub on trial032 to boot 5.14.0-686.el9.x86_64 2026-02-24T18:14:58.897 DEBUG:teuthology.orchestra.run.trial032:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T18:14:58.961 INFO:teuthology.orchestra.run.trial032.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T18:14:58.963 DEBUG:teuthology.orchestra.run.trial032:> sudo command -v grubby && echo yes || echo no 2026-02-24T18:14:59.027 INFO:teuthology.orchestra.run.trial032.stdout:/sbin/grubby 2026-02-24T18:14:59.028 INFO:teuthology.orchestra.run.trial032.stdout:yes 2026-02-24T18:14:59.028 DEBUG:teuthology.orchestra.run.trial032:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T18:14:59.094 INFO:teuthology.orchestra.run.trial032.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T18:14:59.094 DEBUG:teuthology.orchestra.run.trial032:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T18:14:59.167 DEBUG:teuthology.orchestra.run.trial032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T18:14:59.374 INFO:teuthology.orchestra.run.trial032.stderr:Generating grub configuration file ... 2026-02-24T18:14:59.442 INFO:teuthology.orchestra.run.trial032.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T18:14:59.449 INFO:teuthology.orchestra.run.trial032.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T18:14:59.698 INFO:teuthology.orchestra.run.trial032.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T18:14:59.704 INFO:teuthology.orchestra.run.trial032.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T18:14:59.737 INFO:teuthology.orchestra.run.trial032.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T18:14:59.744 INFO:teuthology.orchestra.run.trial032.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T18:14:59.775 INFO:teuthology.orchestra.run.trial032.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T18:14:59.781 INFO:teuthology.orchestra.run.trial032.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T18:14:59.804 INFO:teuthology.orchestra.run.trial032.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T18:14:59.811 INFO:teuthology.orchestra.run.trial032.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T18:14:59.897 INFO:teuthology.orchestra.run.trial032.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:14:59.905 INFO:teuthology.orchestra.run.trial032.stderr:done 2026-02-24T18:14:59.906 DEBUG:teuthology.orchestra.run.trial032:> test -d /sys/firmware/efi 2026-02-24T18:14:59.922 DEBUG:teuthology.orchestra.run.trial032:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T18:14:59.988 INFO:teuthology.orchestra.run.trial032.stdout:/boot/efi/EFI/BOOT 2026-02-24T18:14:59.988 INFO:teuthology.orchestra.run.trial032.stdout:/boot/efi/EFI/centos 2026-02-24T18:14:59.990 DEBUG:teuthology.orchestra.run.trial032:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T18:15:00.054 INFO:teuthology.orchestra.run.trial032.stdout:no 2026-02-24T18:15:00.055 DEBUG:teuthology.orchestra.run.trial032:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T18:15:00.121 INFO:teuthology.orchestra.run.trial032.stdout:yes 2026-02-24T18:15:00.121 DEBUG:teuthology.orchestra.run.trial032:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T18:15:00.184 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial032 2026-02-24T18:15:00.184 DEBUG:teuthology.orchestra.run.trial032:> sudo bash -c ' 2026-02-24T18:15:00.184 DEBUG:teuthology.orchestra.run.trial032:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:15:00.184 DEBUG:teuthology.orchestra.run.trial032:> ' 2026-02-24T18:15:21.785 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-24T18:15:21.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T18:15:21.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:22.407 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T18:15:22.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial018.front.sepia.ceph.com' 2026-02-24T18:15:22.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:24.057 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-24T18:15:24.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial112.front.sepia.ceph.com' 2026-02-24T18:15:24.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:30.226 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-24T18:15:30.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial032.front.sepia.ceph.com' 2026-02-24T18:15:30.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:32.033 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.110 2026-02-24T18:15:32.671 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.18 2026-02-24T18:15:38.367 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.112 2026-02-24T18:15:40.480 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.32 2026-02-24T18:15:41.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T18:15:41.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:41.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial018.front.sepia.ceph.com' 2026-02-24T18:15:41.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:47.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial112.front.sepia.ceph.com' 2026-02-24T18:15:47.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:15:49.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial032.front.sepia.ceph.com' 2026-02-24T18:15:49.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:16:17.111 DEBUG:teuthology.orchestra.run.trial110:> true 2026-02-24T18:16:17.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T18:16:17.338 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-686.el9.x86_64"... 2026-02-24T18:16:17.338 DEBUG:teuthology.orchestra.run.trial110:> uname -r 2026-02-24T18:16:17.398 INFO:teuthology.orchestra.run.trial110.stdout:5.14.0-686.el9.x86_64 2026-02-24T18:16:17.398 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T18:16:17.398 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:16:17.398 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T18:16:17.624 DEBUG:teuthology.orchestra.run.trial018:> true 2026-02-24T18:16:17.839 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial018.front.sepia.ceph.com' 2026-02-24T18:16:17.839 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-24T18:16:17.839 DEBUG:teuthology.orchestra.run.trial018:> uname -r 2026-02-24T18:16:17.896 INFO:teuthology.orchestra.run.trial018.stdout:5.14.0-686.el9.x86_64 2026-02-24T18:16:17.896 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T18:16:17.896 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:16:17.896 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T18:16:18.399 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2026-02-24T18:16:18.399 DEBUG:teuthology.orchestra.run.trial110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:16:18.431 INFO:teuthology.orchestra.run.trial110.stdout:ttyS1 2026-02-24T18:16:18.464 DEBUG:teuthology.parallel:result is None 2026-02-24T18:16:18.897 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T18:16:18.898 DEBUG:teuthology.orchestra.run.trial018:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:16:18.931 INFO:teuthology.orchestra.run.trial018.stdout:ttyS1 2026-02-24T18:16:18.952 DEBUG:teuthology.parallel:result is None 2026-02-24T18:16:23.301 DEBUG:teuthology.orchestra.run.trial112:> true 2026-02-24T18:16:23.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial112.front.sepia.ceph.com' 2026-02-24T18:16:23.472 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-24T18:16:23.472 DEBUG:teuthology.orchestra.run.trial112:> uname -r 2026-02-24T18:16:23.528 INFO:teuthology.orchestra.run.trial112.stdout:5.14.0-686.el9.x86_64 2026-02-24T18:16:23.528 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T18:16:23.528 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:16:23.528 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T18:16:24.529 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-24T18:16:24.529 DEBUG:teuthology.orchestra.run.trial112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:16:24.559 INFO:teuthology.orchestra.run.trial112.stdout:ttyS1 2026-02-24T18:16:24.587 DEBUG:teuthology.parallel:result is None 2026-02-24T18:16:25.355 DEBUG:teuthology.orchestra.run.trial032:> true 2026-02-24T18:16:25.516 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial032.front.sepia.ceph.com' 2026-02-24T18:16:25.516 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-686.el9.x86_64"... 2026-02-24T18:16:25.516 DEBUG:teuthology.orchestra.run.trial032:> uname -r 2026-02-24T18:16:25.568 INFO:teuthology.orchestra.run.trial032.stdout:5.14.0-686.el9.x86_64 2026-02-24T18:16:25.568 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T18:16:25.568 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:16:25.568 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T18:16:26.568 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-24T18:16:26.568 DEBUG:teuthology.orchestra.run.trial032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:16:26.600 INFO:teuthology.orchestra.run.trial032.stdout:ttyS1 2026-02-24T18:16:26.623 DEBUG:teuthology.parallel:result is None 2026-02-24T18:16:26.624 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T18:16:26.636 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T18:16:26.636 DEBUG:teuthology.orchestra.run.trial018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:16:26.639 DEBUG:teuthology.orchestra.run.trial032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:16:26.663 DEBUG:teuthology.orchestra.run.trial110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:16:26.666 DEBUG:teuthology.orchestra.run.trial112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:16:26.684 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T18:16:26.697 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T18:16:26.710 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T18:16:26.710 DEBUG:teuthology.orchestra.run.trial018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:16:26.713 DEBUG:teuthology.orchestra.run.trial032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:16:26.716 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:16:26.725 DEBUG:teuthology.orchestra.run.trial112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:16:26.746 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T18:16:26.759 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T18:16:26.759 DEBUG:teuthology.orchestra.run.trial018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:16:26.768 DEBUG:teuthology.orchestra.run.trial032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:16:26.770 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:16:26.781 DEBUG:teuthology.orchestra.run.trial112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:16:26.790 INFO:teuthology.orchestra.run.trial018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.791 INFO:teuthology.orchestra.run.trial032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.799 INFO:teuthology.orchestra.run.trial018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.800 INFO:teuthology.orchestra.run.trial032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.804 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.805 INFO:teuthology.orchestra.run.trial112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.813 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.814 INFO:teuthology.orchestra.run.trial112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:16:26.816 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T18:16:26.828 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T18:16:26.828 DEBUG:teuthology.orchestra.run.trial018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:16:26.843 DEBUG:teuthology.orchestra.run.trial032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:16:26.845 DEBUG:teuthology.orchestra.run.trial110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:16:26.858 DEBUG:teuthology.orchestra.run.trial112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:16:26.882 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T18:16:26.899 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T18:16:26.899 DEBUG:teuthology.orchestra.run.trial018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:16:26.907 DEBUG:teuthology.orchestra.run.trial032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:16:26.910 DEBUG:teuthology.orchestra.run.trial110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:16:26.922 DEBUG:teuthology.orchestra.run.trial112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:16:26.939 DEBUG:teuthology.orchestra.run.trial018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.060 DEBUG:teuthology.orchestra.run.trial018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.085 DEBUG:teuthology.orchestra.run.trial018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.217 DEBUG:teuthology.orchestra.run.trial018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.242 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:16:27.242 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:16:27.304 DEBUG:teuthology.orchestra.run.trial032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.397 DEBUG:teuthology.orchestra.run.trial032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.423 DEBUG:teuthology.orchestra.run.trial032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.554 DEBUG:teuthology.orchestra.run.trial032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.578 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:16:27.579 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:16:27.639 DEBUG:teuthology.orchestra.run.trial110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.746 DEBUG:teuthology.orchestra.run.trial110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:27.771 DEBUG:teuthology.orchestra.run.trial110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.919 DEBUG:teuthology.orchestra.run.trial110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:27.944 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:16:27.944 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:16:28.009 DEBUG:teuthology.orchestra.run.trial112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:28.091 DEBUG:teuthology.orchestra.run.trial112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:16:28.116 DEBUG:teuthology.orchestra.run.trial112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:28.250 DEBUG:teuthology.orchestra.run.trial112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:16:28.273 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:16:28.273 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:16:28.337 DEBUG:teuthology.orchestra.run.trial018:> sudo service rsyslog restart 2026-02-24T18:16:28.339 DEBUG:teuthology.orchestra.run.trial032:> sudo service rsyslog restart 2026-02-24T18:16:28.341 DEBUG:teuthology.orchestra.run.trial110:> sudo service rsyslog restart 2026-02-24T18:16:28.343 DEBUG:teuthology.orchestra.run.trial112:> sudo service rsyslog restart 2026-02-24T18:16:28.365 INFO:teuthology.orchestra.run.trial018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:16:28.367 INFO:teuthology.orchestra.run.trial032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:16:28.370 INFO:teuthology.orchestra.run.trial110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:16:28.404 INFO:teuthology.orchestra.run.trial112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:16:28.931 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T18:16:28.943 INFO:teuthology.task.internal:Starting timer... 2026-02-24T18:16:28.943 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T18:16:28.960 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T18:16:28.978 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-24T18:16:28.978 DEBUG:teuthology.orchestra.run.trial018:> sudo service auditd rotate 2026-02-24T18:16:29.008 INFO:teuthology.orchestra.run.trial018.stdout:Rotating logs: 2026-02-24T18:16:29.010 DEBUG:teuthology.orchestra.run.trial032:> sudo service auditd rotate 2026-02-24T18:16:29.039 INFO:teuthology.orchestra.run.trial032.stdout:Rotating logs: 2026-02-24T18:16:29.041 DEBUG:teuthology.orchestra.run.trial110:> sudo service auditd rotate 2026-02-24T18:16:29.070 INFO:teuthology.orchestra.run.trial110.stdout:Rotating logs: 2026-02-24T18:16:29.072 DEBUG:teuthology.orchestra.run.trial112:> sudo service auditd rotate 2026-02-24T18:16:29.102 INFO:teuthology.orchestra.run.trial112.stdout:Rotating logs: 2026-02-24T18:16:29.104 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T18:16:29.104 DEBUG:teuthology.orchestra.run.trial018:> /usr/sbin/getenforce 2026-02-24T18:16:29.122 INFO:teuthology.orchestra.run.trial018.stdout:Permissive 2026-02-24T18:16:29.123 DEBUG:teuthology.orchestra.run.trial032:> /usr/sbin/getenforce 2026-02-24T18:16:29.139 INFO:teuthology.orchestra.run.trial032.stdout:Permissive 2026-02-24T18:16:29.140 DEBUG:teuthology.orchestra.run.trial110:> /usr/sbin/getenforce 2026-02-24T18:16:29.156 INFO:teuthology.orchestra.run.trial110.stdout:Permissive 2026-02-24T18:16:29.156 DEBUG:teuthology.orchestra.run.trial112:> /usr/sbin/getenforce 2026-02-24T18:16:29.172 INFO:teuthology.orchestra.run.trial112.stdout:Permissive 2026-02-24T18:16:29.172 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial018.front.sepia.ceph.com': 'permissive', 'ubuntu@trial032.front.sepia.ceph.com': 'permissive', 'ubuntu@trial110.front.sepia.ceph.com': 'permissive', 'ubuntu@trial112.front.sepia.ceph.com': 'permissive'} 2026-02-24T18:16:29.172 DEBUG:teuthology.orchestra.run.trial018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:16:29.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:16:29.197 DEBUG:teuthology.orchestra.run.trial032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:16:29.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:16:29.223 DEBUG:teuthology.orchestra.run.trial110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:16:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:16:29.247 DEBUG:teuthology.orchestra.run.trial112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:16:29.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:16:29.273 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T18:16:29.273 DEBUG:teuthology.orchestra.run.trial018:> sudo /usr/sbin/setenforce permissive 2026-02-24T18:16:29.298 DEBUG:teuthology.orchestra.run.trial032:> sudo /usr/sbin/setenforce permissive 2026-02-24T18:16:29.323 DEBUG:teuthology.orchestra.run.trial110:> sudo /usr/sbin/setenforce permissive 2026-02-24T18:16:29.352 DEBUG:teuthology.orchestra.run.trial112:> sudo /usr/sbin/setenforce permissive 2026-02-24T18:16:29.382 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T18:16:29.400 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T18:16:29.411 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T18:16:29.413 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial018.front.sepia.ceph.com,trial032.front.sepia.ceph.com,trial110.front.sepia.ceph.com,trial112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T18:19:17.666 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial018.front.sepia.ceph.com'), Remote(name='ubuntu@trial032.front.sepia.ceph.com'), Remote(name='ubuntu@trial110.front.sepia.ceph.com'), Remote(name='ubuntu@trial112.front.sepia.ceph.com')] 2026-02-24T18:19:17.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial018.front.sepia.ceph.com' 2026-02-24T18:19:17.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:19:17.779 DEBUG:teuthology.orchestra.run.trial018:> true 2026-02-24T18:19:17.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial018.front.sepia.ceph.com' 2026-02-24T18:19:17.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial032.front.sepia.ceph.com' 2026-02-24T18:19:17.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:19:17.942 DEBUG:teuthology.orchestra.run.trial032:> true 2026-02-24T18:19:18.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial032.front.sepia.ceph.com' 2026-02-24T18:19:18.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T18:19:18.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:19:18.103 DEBUG:teuthology.orchestra.run.trial110:> true 2026-02-24T18:19:18.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T18:19:18.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial112.front.sepia.ceph.com' 2026-02-24T18:19:18.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:19:18.260 DEBUG:teuthology.orchestra.run.trial112:> true 2026-02-24T18:19:18.337 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial112.front.sepia.ceph.com' 2026-02-24T18:19:18.339 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T18:19:18.356 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T18:19:18.356 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:19:18.357 DEBUG:teuthology.orchestra.run.trial018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:19:18.359 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:19:18.359 DEBUG:teuthology.orchestra.run.trial032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:19:18.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:19:18.362 DEBUG:teuthology.orchestra.run.trial110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:19:18.363 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:19:18.364 DEBUG:teuthology.orchestra.run.trial112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:19:18.386 INFO:teuthology.orchestra.run.trial018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T18:19:18.388 INFO:teuthology.orchestra.run.trial032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T18:19:18.391 INFO:teuthology.orchestra.run.trial110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T18:19:18.398 INFO:teuthology.orchestra.run.trial018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T18:19:18.400 INFO:teuthology.orchestra.run.trial032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T18:19:18.403 INFO:teuthology.orchestra.run.trial110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T18:19:18.404 INFO:teuthology.orchestra.run.trial112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T18:19:18.416 INFO:teuthology.orchestra.run.trial112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T18:19:18.419 INFO:teuthology.orchestra.run.trial018.stderr:sudo: ntpd: command not found 2026-02-24T18:19:18.420 INFO:teuthology.orchestra.run.trial032.stderr:sudo: ntpd: command not found 2026-02-24T18:19:18.424 INFO:teuthology.orchestra.run.trial110.stderr:sudo: ntpd: command not found 2026-02-24T18:19:18.429 INFO:teuthology.orchestra.run.trial018.stdout:506 Cannot talk to daemon 2026-02-24T18:19:18.430 INFO:teuthology.orchestra.run.trial032.stdout:506 Cannot talk to daemon 2026-02-24T18:19:18.434 INFO:teuthology.orchestra.run.trial110.stdout:506 Cannot talk to daemon 2026-02-24T18:19:18.437 INFO:teuthology.orchestra.run.trial112.stderr:sudo: ntpd: command not found 2026-02-24T18:19:18.440 INFO:teuthology.orchestra.run.trial018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T18:19:18.441 INFO:teuthology.orchestra.run.trial032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T18:19:18.445 INFO:teuthology.orchestra.run.trial110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T18:19:18.446 INFO:teuthology.orchestra.run.trial112.stdout:506 Cannot talk to daemon 2026-02-24T18:19:18.453 INFO:teuthology.orchestra.run.trial018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T18:19:18.454 INFO:teuthology.orchestra.run.trial032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T18:19:18.457 INFO:teuthology.orchestra.run.trial110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T18:19:18.458 INFO:teuthology.orchestra.run.trial112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T18:19:18.470 INFO:teuthology.orchestra.run.trial112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T18:19:18.497 INFO:teuthology.orchestra.run.trial032.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:19:18.504 INFO:teuthology.orchestra.run.trial018.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:19:18.507 INFO:teuthology.orchestra.run.trial110.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:19:18.519 INFO:teuthology.orchestra.run.trial112.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:19:18.520 INFO:teuthology.orchestra.run.trial032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:19:18.521 INFO:teuthology.orchestra.run.trial032.stdout:=============================================================================== 2026-02-24T18:19:18.521 INFO:teuthology.orchestra.run.trial032.stdout:^? 172-104-209-204.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.521 INFO:teuthology.orchestra.run.trial032.stdout:^? chi3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.521 INFO:teuthology.orchestra.run.trial032.stdout:^? arm1.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.521 INFO:teuthology.orchestra.run.trial032.stdout:^? us-west-1.clearnet.pw 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:=============================================================================== 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:^? vps5.drown.org 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:^? linode1.ernest-doss.org 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:^? 198.71.50.75 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.526 INFO:teuthology.orchestra.run.trial018.stdout:^? us-east-2.clearnet.pw 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:=============================================================================== 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:^? jane.qotw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:^? vps6.drown.org 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:^? 45.55.126.202 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.527 INFO:teuthology.orchestra.run.trial110.stdout:^? time.cloudflare.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:=============================================================================== 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:^? 66.85.78.80 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:^? 50.218.103.254 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:^? 172-233-157-223.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.534 INFO:teuthology.orchestra.run.trial112.stdout:^? ntp4.glypnod.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-24T18:19:18.534 INFO:teuthology.run_tasks:Running task install... 2026-02-24T18:19:18.546 DEBUG:teuthology.task.install:project ceph 2026-02-24T18:19:18.546 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-24T18:19:18.546 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-24T18:19:18.546 INFO:teuthology.task.install:Using flavor: default 2026-02-24T18:19:18.550 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-24T18:19:18.550 INFO:teuthology.task.install:extra packages: [] 2026-02-24T18:19:18.550 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-24T18:19:18.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:19:18.552 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-24T18:19:18.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:19:18.553 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-24T18:19:18.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:19:18.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-24T18:19:18.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:19:18.832 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-24T18:19:18.832 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-24T18:19:18.844 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-24T18:19:18.844 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-24T18:19:18.844 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-24T18:19:18.844 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-24T18:19:18.845 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-24T18:19:18.845 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-24T18:19:19.210 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T18:19:19.210 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:19:19.210 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T18:19:19.212 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T18:19:19.212 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:19:19.212 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T18:19:19.215 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T18:19:19.215 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:19:19.216 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T18:19:19.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-24T18:19:19.237 DEBUG:teuthology.orchestra.run.trial112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T18:19:19.238 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-24T18:19:19.239 DEBUG:teuthology.orchestra.run.trial110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T18:19:19.242 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-24T18:19:19.243 DEBUG:teuthology.orchestra.run.trial018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T18:19:19.268 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T18:19:19.268 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:19:19.268 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T18:19:19.295 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-24T18:19:19.295 DEBUG:teuthology.orchestra.run.trial032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T18:19:19.302 DEBUG:teuthology.orchestra.run.trial112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T18:19:19.304 DEBUG:teuthology.orchestra.run.trial110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T18:19:19.308 DEBUG:teuthology.orchestra.run.trial018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T18:19:19.360 DEBUG:teuthology.orchestra.run.trial032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T18:19:19.374 DEBUG:teuthology.orchestra.run.trial112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T18:19:19.376 DEBUG:teuthology.orchestra.run.trial110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T18:19:19.380 DEBUG:teuthology.orchestra.run.trial018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T18:19:19.432 DEBUG:teuthology.orchestra.run.trial032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T18:19:19.438 INFO:teuthology.orchestra.run.trial112.stdout:check_obsoletes = 1 2026-02-24T18:19:19.438 INFO:teuthology.orchestra.run.trial110.stdout:check_obsoletes = 1 2026-02-24T18:19:19.439 DEBUG:teuthology.orchestra.run.trial112:> sudo yum clean all 2026-02-24T18:19:19.440 DEBUG:teuthology.orchestra.run.trial110:> sudo yum clean all 2026-02-24T18:19:19.444 INFO:teuthology.orchestra.run.trial018.stdout:check_obsoletes = 1 2026-02-24T18:19:19.445 DEBUG:teuthology.orchestra.run.trial018:> sudo yum clean all 2026-02-24T18:19:19.495 INFO:teuthology.orchestra.run.trial032.stdout:check_obsoletes = 1 2026-02-24T18:19:19.496 DEBUG:teuthology.orchestra.run.trial032:> sudo yum clean all 2026-02-24T18:19:19.586 INFO:teuthology.orchestra.run.trial110.stdout:41 files removed 2026-02-24T18:19:19.589 INFO:teuthology.orchestra.run.trial112.stdout:41 files removed 2026-02-24T18:19:19.593 INFO:teuthology.orchestra.run.trial018.stdout:41 files removed 2026-02-24T18:19:19.599 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-radosgw 2026-02-24T18:19:19.603 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-radosgw 2026-02-24T18:19:19.607 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-radosgw 2026-02-24T18:19:19.644 INFO:teuthology.orchestra.run.trial032.stdout:41 files removed 2026-02-24T18:19:19.657 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-radosgw 2026-02-24T18:19:20.343 INFO:teuthology.orchestra.run.trial110.stdout:ceph packages for x86_64 144 kB/s | 86 kB 00:00 2026-02-24T18:19:20.352 INFO:teuthology.orchestra.run.trial018.stdout:ceph packages for x86_64 143 kB/s | 86 kB 00:00 2026-02-24T18:19:20.354 INFO:teuthology.orchestra.run.trial112.stdout:ceph packages for x86_64 142 kB/s | 86 kB 00:00 2026-02-24T18:19:20.398 INFO:teuthology.orchestra.run.trial032.stdout:ceph packages for x86_64 144 kB/s | 86 kB 00:00 2026-02-24T18:19:20.860 INFO:teuthology.orchestra.run.trial110.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2026-02-24T18:19:20.863 INFO:teuthology.orchestra.run.trial018.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2026-02-24T18:19:20.870 INFO:teuthology.orchestra.run.trial112.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2026-02-24T18:19:20.913 INFO:teuthology.orchestra.run.trial032.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2026-02-24T18:19:21.430 INFO:teuthology.orchestra.run.trial112.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2026-02-24T18:19:21.454 INFO:teuthology.orchestra.run.trial110.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2026-02-24T18:19:21.455 INFO:teuthology.orchestra.run.trial018.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2026-02-24T18:19:21.483 INFO:teuthology.orchestra.run.trial032.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2026-02-24T18:19:22.046 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.9 MB 00:00 2026-02-24T18:19:22.891 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.9 MB 00:01 2026-02-24T18:19:22.902 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.9 MB 00:01 2026-02-24T18:19:23.185 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.9 MB 00:01 2026-02-24T18:19:24.114 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - AppStream 66 MB/s | 27 MB 00:00 2026-02-24T18:19:25.095 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-24T18:19:25.282 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - AppStream 15 MB/s | 27 MB 00:01 2026-02-24T18:19:26.457 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 27 MB 00:03 2026-02-24T18:19:29.078 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 8.0 MB 00:01 2026-02-24T18:19:29.287 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 8.0 MB 00:01 2026-02-24T18:19:30.228 INFO:teuthology.orchestra.run.trial032.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 20 kB 00:00 2026-02-24T18:19:30.667 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 8.0 MB 00:01 2026-02-24T18:19:30.720 INFO:teuthology.orchestra.run.trial018.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 20 kB 00:00 2026-02-24T18:19:32.240 INFO:teuthology.orchestra.run.trial112.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 20 kB 00:00 2026-02-24T18:19:32.421 INFO:teuthology.orchestra.run.trial018.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2026-02-24T18:19:32.836 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 8.0 MB 00:06 2026-02-24T18:19:32.873 INFO:teuthology.orchestra.run.trial112.stdout:Extra Packages for Enterprise Linux 36 MB/s | 20 MB 00:00 2026-02-24T18:19:34.027 INFO:teuthology.orchestra.run.trial110.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 20 kB 00:00 2026-02-24T18:19:34.268 INFO:teuthology.orchestra.run.trial110.stdout:Extra Packages for Enterprise Linux 115 MB/s | 20 MB 00:00 2026-02-24T18:19:35.679 INFO:teuthology.orchestra.run.trial018.stdout:lab-extras 1.4 MB/s | 50 kB 00:00 2026-02-24T18:19:36.059 INFO:teuthology.orchestra.run.trial112.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-24T18:19:36.901 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repository Size 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout:Upgrading: 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-24T18:19:36.903 INFO:teuthology.orchestra.run.trial018.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout:Installing dependencies: 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-24T18:19:36.904 INFO:teuthology.orchestra.run.trial018.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T18:19:36.905 INFO:teuthology.orchestra.run.trial018.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T18:19:36.906 INFO:teuthology.orchestra.run.trial018.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout:Installing weak dependencies: 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout:Install 33 Packages 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout:Upgrade 2 Packages 2026-02-24T18:19:36.907 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:36.908 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 63 M 2026-02-24T18:19:36.908 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:19:37.288 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:19:37.289 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:37.289 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repository Size 2026-02-24T18:19:37.289 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:37.289 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout:Upgrading: 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout:Installing dependencies: 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T18:19:37.290 INFO:teuthology.orchestra.run.trial112.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T18:19:37.291 INFO:teuthology.orchestra.run.trial112.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T18:19:37.292 INFO:teuthology.orchestra.run.trial112.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout:Installing weak dependencies: 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout:Install 33 Packages 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout:Upgrade 2 Packages 2026-02-24T18:19:37.293 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:37.294 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 63 M 2026-02-24T18:19:37.294 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:19:37.407 INFO:teuthology.orchestra.run.trial110.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-24T18:19:37.425 INFO:teuthology.orchestra.run.trial032.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 20 MB 00:07 2026-02-24T18:19:37.991 INFO:teuthology.orchestra.run.trial018.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 9.8 MB/s | 5.5 MB 00:00 2026-02-24T18:19:38.024 INFO:teuthology.orchestra.run.trial018.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 743 kB/s | 25 kB 00:00 2026-02-24T18:19:38.113 INFO:teuthology.orchestra.run.trial018.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 11 MB/s | 999 kB 00:00 2026-02-24T18:19:38.162 INFO:teuthology.orchestra.run.trial018.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 10 MB/s | 503 kB 00:00 2026-02-24T18:19:38.609 INFO:teuthology.orchestra.run.trial018.stdout:(5/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_64 12 MB/s | 5.4 MB 00:00 2026-02-24T18:19:38.623 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repository Size 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout:Upgrading: 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-24T18:19:38.624 INFO:teuthology.orchestra.run.trial110.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout:Installing dependencies: 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-24T18:19:38.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Installing weak dependencies: 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Install 33 Packages 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Upgrade 2 Packages 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 63 M 2026-02-24T18:19:38.626 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:19:38.662 INFO:teuthology.orchestra.run.trial018.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 859 kB/s | 45 kB 00:00 2026-02-24T18:19:38.723 INFO:teuthology.orchestra.run.trial018.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 2.3 MB/s | 142 kB 00:00 2026-02-24T18:19:38.788 INFO:teuthology.orchestra.run.trial018.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 2.5 MB/s | 165 kB 00:00 2026-02-24T18:19:39.053 INFO:teuthology.orchestra.run.trial018.stdout:(9/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 6.6 MB/s | 11 MB 00:01 2026-02-24T18:19:39.075 INFO:teuthology.orchestra.run.trial112.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 4.3 MB/s | 5.5 MB 00:01 2026-02-24T18:19:39.109 INFO:teuthology.orchestra.run.trial018.stdout:(10/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 5.3 MB/s | 303 kB 00:00 2026-02-24T18:19:39.130 INFO:teuthology.orchestra.run.trial112.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 460 kB/s | 25 kB 00:00 2026-02-24T18:19:39.163 INFO:teuthology.orchestra.run.trial018.stdout:(11/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 1.8 MB/s | 100 kB 00:00 2026-02-24T18:19:39.340 INFO:teuthology.orchestra.run.trial018.stdout:(12/35): python3-rados-19.2.3-689.gf90f4eae.el9 585 kB/s | 322 kB 00:00 2026-02-24T18:19:39.361 INFO:teuthology.orchestra.run.trial112.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 4.2 MB/s | 999 kB 00:00 2026-02-24T18:19:39.427 INFO:teuthology.orchestra.run.trial018.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 126 kB/s | 33 kB 00:00 2026-02-24T18:19:39.482 INFO:teuthology.orchestra.run.trial112.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 4.1 MB/s | 503 kB 00:00 2026-02-24T18:19:39.665 INFO:teuthology.orchestra.run.trial018.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 392 kB/s | 93 kB 00:00 2026-02-24T18:19:39.735 INFO:teuthology.orchestra.run.trial018.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 1.4 MB/s | 104 kB 00:00 2026-02-24T18:19:39.779 INFO:teuthology.orchestra.run.trial018.stdout:(16/35): libnbd-1.20.3-4.el9.x86_64.rpm 3.7 MB/s | 164 kB 00:00 2026-02-24T18:19:39.803 INFO:teuthology.orchestra.run.trial018.stdout:(17/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.4 MB/s | 160 kB 00:00 2026-02-24T18:19:39.825 INFO:teuthology.orchestra.run.trial018.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-24T18:19:39.870 INFO:teuthology.orchestra.run.trial018.stdout:(19/35): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 662 kB 00:00 2026-02-24T18:19:39.894 INFO:teuthology.orchestra.run.trial018.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-24T18:19:39.918 INFO:teuthology.orchestra.run.trial018.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 10 MB/s | 240 kB 00:00 2026-02-24T18:19:39.939 INFO:teuthology.orchestra.run.trial018.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 1.7 MB/s | 37 kB 00:00 2026-02-24T18:19:39.961 INFO:teuthology.orchestra.run.trial018.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.0 MB/s | 66 kB 00:00 2026-02-24T18:19:40.015 INFO:teuthology.orchestra.run.trial018.stdout:(24/35): nvme-cli-2.16-1.el9.x86_64.rpm 1.7 MB/s | 1.2 MB 00:00 2026-02-24T18:19:40.137 INFO:teuthology.orchestra.run.trial018.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.7 MB/s | 308 kB 00:00 2026-02-24T18:19:40.166 INFO:teuthology.orchestra.run.trial018.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 870 kB/s | 25 kB 00:00 2026-02-24T18:19:40.195 INFO:teuthology.orchestra.run.trial018.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 1.7 MB/s | 49 kB 00:00 2026-02-24T18:19:40.224 INFO:teuthology.orchestra.run.trial018.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 2.3 MB/s | 67 kB 00:00 2026-02-24T18:19:40.281 INFO:teuthology.orchestra.run.trial018.stdout:(29/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2026-02-24T18:19:40.313 INFO:teuthology.orchestra.run.trial018.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 5.9 MB/s | 191 kB 00:00 2026-02-24T18:19:40.364 INFO:teuthology.orchestra.run.trial018.stdout:(31/35): libarrow-9.0.0-15.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2026-02-24T18:19:40.401 INFO:teuthology.orchestra.run.trial018.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2026-02-24T18:19:40.499 INFO:teuthology.orchestra.run.trial018.stdout:(33/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 7.1 MB/s | 22 MB 00:03 2026-02-24T18:19:40.535 INFO:teuthology.orchestra.run.trial112.stdout:(5/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_64 5.1 MB/s | 5.4 MB 00:01 2026-02-24T18:19:40.597 INFO:teuthology.orchestra.run.trial112.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 728 kB/s | 45 kB 00:00 2026-02-24T18:19:40.659 INFO:teuthology.orchestra.run.trial112.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 2.2 MB/s | 142 kB 00:00 2026-02-24T18:19:40.672 INFO:teuthology.orchestra.run.trial032.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-24T18:19:41.178 INFO:teuthology.orchestra.run.trial112.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 318 kB/s | 165 kB 00:00 2026-02-24T18:19:41.284 INFO:teuthology.orchestra.run.trial018.stdout:(34/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 3.6 MB/s | 3.2 MB 00:00 2026-02-24T18:19:41.302 INFO:teuthology.orchestra.run.trial112.stdout:(9/35): python3-rados-19.2.3-689.gf90f4eae.el9. 2.5 MB/s | 322 kB 00:00 2026-02-24T18:19:41.379 INFO:teuthology.orchestra.run.trial110.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 2.4 MB/s | 5.5 MB 00:02 2026-02-24T18:19:41.419 INFO:teuthology.orchestra.run.trial112.stdout:(10/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 2.5 MB/s | 303 kB 00:00 2026-02-24T18:19:41.433 INFO:teuthology.orchestra.run.trial110.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 466 kB/s | 25 kB 00:00 2026-02-24T18:19:41.476 INFO:teuthology.orchestra.run.trial112.stdout:(11/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 1.7 MB/s | 100 kB 00:00 2026-02-24T18:19:41.785 INFO:teuthology.orchestra.run.trial110.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 2.8 MB/s | 999 kB 00:00 2026-02-24T18:19:41.878 INFO:teuthology.orchestra.run.trial112.stdout:(12/35): mailcap-2.1.49-5.el9.noarch.rpm 82 kB/s | 33 kB 00:00 2026-02-24T18:19:41.898 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repository Size 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:19:41.900 INFO:teuthology.orchestra.run.trial032.stdout:Upgrading: 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout:Installing dependencies: 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-24T18:19:41.901 INFO:teuthology.orchestra.run.trial032.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-24T18:19:41.902 INFO:teuthology.orchestra.run.trial032.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout:Installing weak dependencies: 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:41.903 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:19:41.904 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:41.904 INFO:teuthology.orchestra.run.trial032.stdout:Install 33 Packages 2026-02-24T18:19:41.904 INFO:teuthology.orchestra.run.trial032.stdout:Upgrade 2 Packages 2026-02-24T18:19:41.904 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:41.904 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 63 M 2026-02-24T18:19:41.905 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:19:41.964 INFO:teuthology.orchestra.run.trial110.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 2.7 MB/s | 503 kB 00:00 2026-02-24T18:19:42.029 INFO:teuthology.orchestra.run.trial110.stdout:(5/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 3.7 MB/s | 11 MB 00:02 2026-02-24T18:19:42.087 INFO:teuthology.orchestra.run.trial110.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 774 kB/s | 45 kB 00:00 2026-02-24T18:19:42.150 INFO:teuthology.orchestra.run.trial110.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 2.2 MB/s | 142 kB 00:00 2026-02-24T18:19:42.215 INFO:teuthology.orchestra.run.trial110.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 2.5 MB/s | 165 kB 00:00 2026-02-24T18:19:42.273 INFO:teuthology.orchestra.run.trial112.stdout:(13/35): nvme-cli-2.16-1.el9.x86_64.rpm 2.9 MB/s | 1.2 MB 00:00 2026-02-24T18:19:42.353 INFO:teuthology.orchestra.run.trial110.stdout:(9/35): python3-rados-19.2.3-689.gf90f4eae.el9. 2.3 MB/s | 322 kB 00:00 2026-02-24T18:19:42.355 INFO:teuthology.orchestra.run.trial112.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 1.1 MB/s | 93 kB 00:00 2026-02-24T18:19:42.378 INFO:teuthology.orchestra.run.trial112.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 4.5 MB/s | 104 kB 00:00 2026-02-24T18:19:42.387 INFO:teuthology.orchestra.run.trial112.stdout:(16/35): libnbd-1.20.3-4.el9.x86_64.rpm 18 MB/s | 164 kB 00:00 2026-02-24T18:19:42.394 INFO:teuthology.orchestra.run.trial112.stdout:(17/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 23 MB/s | 160 kB 00:00 2026-02-24T18:19:42.398 INFO:teuthology.orchestra.run.trial112.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 10 MB/s | 45 kB 00:00 2026-02-24T18:19:42.408 INFO:teuthology.orchestra.run.trial112.stdout:(19/35): librdkafka-1.6.1-102.el9.x86_64.rpm 65 MB/s | 662 kB 00:00 2026-02-24T18:19:42.415 INFO:teuthology.orchestra.run.trial112.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 45 MB/s | 292 kB 00:00 2026-02-24T18:19:42.421 INFO:teuthology.orchestra.run.trial112.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 39 MB/s | 240 kB 00:00 2026-02-24T18:19:42.425 INFO:teuthology.orchestra.run.trial112.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 8.6 MB/s | 37 kB 00:00 2026-02-24T18:19:42.430 INFO:teuthology.orchestra.run.trial112.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 15 MB/s | 66 kB 00:00 2026-02-24T18:19:42.553 INFO:teuthology.orchestra.run.trial110.stdout:(10/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 1.5 MB/s | 303 kB 00:00 2026-02-24T18:19:42.614 INFO:teuthology.orchestra.run.trial110.stdout:(11/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 1.6 MB/s | 100 kB 00:00 2026-02-24T18:19:42.717 INFO:teuthology.orchestra.run.trial018.stdout:(35/35): librados2-19.2.3-689.gf90f4eae.el9.x86 1.5 MB/s | 3.4 MB 00:02 2026-02-24T18:19:42.719 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:19:42.719 INFO:teuthology.orchestra.run.trial018.stdout:Total 11 MB/s | 63 MB 00:05 2026-02-24T18:19:42.779 INFO:teuthology.orchestra.run.trial112.stdout:(24/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 881 kB/s | 308 kB 00:00 2026-02-24T18:19:42.823 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:19:42.842 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:19:42.842 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:19:43.051 INFO:teuthology.orchestra.run.trial112.stdout:(25/35): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-24T18:19:43.092 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:19:43.092 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:19:43.103 INFO:teuthology.orchestra.run.trial112.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 475 kB/s | 25 kB 00:00 2026-02-24T18:19:43.157 INFO:teuthology.orchestra.run.trial112.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 913 kB/s | 49 kB 00:00 2026-02-24T18:19:43.172 INFO:teuthology.orchestra.run.trial110.stdout:(12/35): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-24T18:19:43.213 INFO:teuthology.orchestra.run.trial112.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2026-02-24T18:19:43.277 INFO:teuthology.orchestra.run.trial112.stdout:(29/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 13 MB/s | 838 kB 00:00 2026-02-24T18:19:43.334 INFO:teuthology.orchestra.run.trial112.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 3.3 MB/s | 191 kB 00:00 2026-02-24T18:19:43.393 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:19:43.401 INFO:teuthology.orchestra.run.trial018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-24T18:19:43.402 INFO:teuthology.orchestra.run.trial112.stdout:(31/35): thrift-0.15.0-4.el9.x86_64.rpm 23 MB/s | 1.6 MB 00:00 2026-02-24T18:19:43.404 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-24T18:19:43.415 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-24T18:19:43.565 INFO:teuthology.orchestra.run.trial018.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-24T18:19:43.569 INFO:teuthology.orchestra.run.trial018.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:43.600 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:43.643 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:43.643 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:43.656 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:43.660 INFO:teuthology.orchestra.run.trial018.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-24T18:19:43.671 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-24T18:19:43.693 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:43.694 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:43.708 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:43.740 INFO:teuthology.orchestra.run.trial018.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-24T18:19:43.744 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-24T18:19:43.750 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-24T18:19:43.753 INFO:teuthology.orchestra.run.trial018.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-24T18:19:43.756 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-24T18:19:43.760 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-24T18:19:43.803 INFO:teuthology.orchestra.run.trial110.stdout:(13/35): nvme-cli-2.16-1.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2026-02-24T18:19:43.899 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:43.900 INFO:teuthology.orchestra.run.trial018.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:43.925 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:43.929 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-24T18:19:43.935 INFO:teuthology.orchestra.run.trial018.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-24T18:19:43.962 INFO:teuthology.orchestra.run.trial110.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 586 kB/s | 93 kB 00:00 2026-02-24T18:19:43.981 INFO:teuthology.orchestra.run.trial110.stdout:(15/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_6 2.7 MB/s | 5.4 MB 00:02 2026-02-24T18:19:44.121 INFO:teuthology.orchestra.run.trial110.stdout:(16/35): boost-program-options-1.75.0-13.el9.x8 654 kB/s | 104 kB 00:00 2026-02-24T18:19:44.159 INFO:teuthology.orchestra.run.trial018.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-24T18:19:44.160 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:44.176 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:44.180 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-24T18:19:44.185 INFO:teuthology.orchestra.run.trial018.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-24T18:19:44.188 INFO:teuthology.orchestra.run.trial018.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-24T18:19:44.189 INFO:teuthology.orchestra.run.trial018.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:44.205 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:44.218 INFO:teuthology.orchestra.run.trial110.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 691 kB/s | 164 kB 00:00 2026-02-24T18:19:44.321 INFO:teuthology.orchestra.run.trial018.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-24T18:19:44.323 INFO:teuthology.orchestra.run.trial018.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:44.376 INFO:teuthology.orchestra.run.trial110.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 286 kB/s | 45 kB 00:00 2026-02-24T18:19:44.494 INFO:teuthology.orchestra.run.trial112.stdout:(32/35): librados2-19.2.3-689.gf90f4eae.el9.x86 3.1 MB/s | 3.4 MB 00:01 2026-02-24T18:19:44.537 INFO:teuthology.orchestra.run.trial110.stdout:(19/35): librdkafka-1.6.1-102.el9.x86_64.rpm 4.0 MB/s | 662 kB 00:00 2026-02-24T18:19:44.610 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:44.610 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:19:44.610 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:44.646 INFO:teuthology.orchestra.run.trial018.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-24T18:19:44.648 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-24T18:19:44.697 INFO:teuthology.orchestra.run.trial110.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2026-02-24T18:19:44.749 INFO:teuthology.orchestra.run.trial110.stdout:(21/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 255 kB/s | 160 kB 00:00 2026-02-24T18:19:44.808 INFO:teuthology.orchestra.run.trial032.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 2.5 MB/s | 5.5 MB 00:02 2026-02-24T18:19:44.857 INFO:teuthology.orchestra.run.trial110.stdout:(22/35): qatlib-25.08.0-2.el9.x86_64.rpm 1.5 MB/s | 240 kB 00:00 2026-02-24T18:19:44.870 INFO:teuthology.orchestra.run.trial032.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 411 kB/s | 25 kB 00:00 2026-02-24T18:19:44.906 INFO:teuthology.orchestra.run.trial110.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 236 kB/s | 37 kB 00:00 2026-02-24T18:19:45.015 INFO:teuthology.orchestra.run.trial110.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 420 kB/s | 66 kB 00:00 2026-02-24T18:19:45.184 INFO:teuthology.orchestra.run.trial112.stdout:(33/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 4.6 MB/s | 3.2 MB 00:00 2026-02-24T18:19:45.189 INFO:teuthology.orchestra.run.trial032.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 3.1 MB/s | 999 kB 00:00 2026-02-24T18:19:45.369 INFO:teuthology.orchestra.run.trial032.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 2.7 MB/s | 503 kB 00:00 2026-02-24T18:19:45.378 INFO:teuthology.orchestra.run.trial110.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 653 kB/s | 308 kB 00:00 2026-02-24T18:19:45.450 INFO:teuthology.orchestra.run.trial110.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 344 kB/s | 25 kB 00:00 2026-02-24T18:19:45.526 INFO:teuthology.orchestra.run.trial110.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2026-02-24T18:19:45.604 INFO:teuthology.orchestra.run.trial110.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 858 kB/s | 67 kB 00:00 2026-02-24T18:19:45.629 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:45.632 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:45.748 INFO:teuthology.orchestra.run.trial110.stdout:(29/35): libarrow-9.0.0-15.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2026-02-24T18:19:45.803 INFO:teuthology.orchestra.run.trial110.stdout:(30/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2026-02-24T18:19:45.819 INFO:teuthology.orchestra.run.trial110.stdout:(31/35): re2-20211101-20.el9.x86_64.rpm 2.6 MB/s | 191 kB 00:00 2026-02-24T18:19:45.912 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:45.916 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:45.954 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:45.955 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T18:19:45.955 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T18:19:45.955 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:45.958 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:45.971 INFO:teuthology.orchestra.run.trial110.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 9.4 MB/s | 1.6 MB 00:00 2026-02-24T18:19:46.558 INFO:teuthology.orchestra.run.trial032.stdout:(5/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 2.7 MB/s | 11 MB 00:03 2026-02-24T18:19:46.568 INFO:teuthology.orchestra.run.trial110.stdout:(33/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 2.9 MB/s | 22 MB 00:07 2026-02-24T18:19:46.609 INFO:teuthology.orchestra.run.trial032.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 876 kB/s | 45 kB 00:00 2026-02-24T18:19:46.665 INFO:teuthology.orchestra.run.trial032.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 2.5 MB/s | 142 kB 00:00 2026-02-24T18:19:46.722 INFO:teuthology.orchestra.run.trial032.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 2.8 MB/s | 165 kB 00:00 2026-02-24T18:19:46.834 INFO:teuthology.orchestra.run.trial112.stdout:(34/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9. 1.2 MB/s | 11 MB 00:09 2026-02-24T18:19:46.835 INFO:teuthology.orchestra.run.trial032.stdout:(9/35): python3-rados-19.2.3-689.gf90f4eae.el9. 2.8 MB/s | 322 kB 00:00 2026-02-24T18:19:46.885 INFO:teuthology.orchestra.run.trial032.stdout:(10/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_6 3.6 MB/s | 5.4 MB 00:01 2026-02-24T18:19:46.940 INFO:teuthology.orchestra.run.trial032.stdout:(11/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 1.8 MB/s | 100 kB 00:00 2026-02-24T18:19:46.941 INFO:teuthology.orchestra.run.trial032.stdout:(12/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 2.8 MB/s | 303 kB 00:00 2026-02-24T18:19:47.201 INFO:teuthology.orchestra.run.trial110.stdout:(34/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 2.6 MB/s | 3.2 MB 00:01 2026-02-24T18:19:47.228 INFO:teuthology.orchestra.run.trial110.stdout:(35/35): librados2-19.2.3-689.gf90f4eae.el9.x86 2.4 MB/s | 3.4 MB 00:01 2026-02-24T18:19:47.231 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:19:47.231 INFO:teuthology.orchestra.run.trial110.stdout:Total 7.3 MB/s | 63 MB 00:08 2026-02-24T18:19:47.327 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:19:47.346 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:19:47.346 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:19:47.494 INFO:teuthology.orchestra.run.trial032.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 60 kB/s | 33 kB 00:00 2026-02-24T18:19:47.596 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:19:47.596 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:19:47.729 INFO:teuthology.orchestra.run.trial032.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 397 kB/s | 93 kB 00:00 2026-02-24T18:19:47.815 INFO:teuthology.orchestra.run.trial032.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 1.2 MB/s | 104 kB 00:00 2026-02-24T18:19:47.889 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:19:47.892 INFO:teuthology.orchestra.run.trial032.stdout:(16/35): nvme-cli-2.16-1.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:00 2026-02-24T18:19:47.897 INFO:teuthology.orchestra.run.trial032.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 1.9 MB/s | 164 kB 00:00 2026-02-24T18:19:47.898 INFO:teuthology.orchestra.run.trial110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-24T18:19:47.900 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-24T18:19:47.912 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-24T18:19:47.976 INFO:teuthology.orchestra.run.trial032.stdout:(18/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2026-02-24T18:19:47.980 INFO:teuthology.orchestra.run.trial032.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 551 kB/s | 45 kB 00:00 2026-02-24T18:19:48.061 INFO:teuthology.orchestra.run.trial032.stdout:(20/35): librdkafka-1.6.1-102.el9.x86_64.rpm 7.6 MB/s | 662 kB 00:00 2026-02-24T18:19:48.062 INFO:teuthology.orchestra.run.trial110.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-24T18:19:48.063 INFO:teuthology.orchestra.run.trial032.stdout:(21/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2026-02-24T18:19:48.067 INFO:teuthology.orchestra.run.trial110.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:48.097 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:48.139 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:48.139 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:48.145 INFO:teuthology.orchestra.run.trial032.stdout:(22/35): qatlib-25.08.0-2.el9.x86_64.rpm 2.8 MB/s | 240 kB 00:00 2026-02-24T18:19:48.148 INFO:teuthology.orchestra.run.trial032.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 435 kB/s | 37 kB 00:00 2026-02-24T18:19:48.151 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:48.155 INFO:teuthology.orchestra.run.trial110.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-24T18:19:48.164 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-24T18:19:48.186 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:48.187 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:48.199 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:48.220 INFO:teuthology.orchestra.run.trial032.stdout:(24/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.2 MB/s | 308 kB 00:00 2026-02-24T18:19:48.227 INFO:teuthology.orchestra.run.trial032.stdout:(25/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 803 kB/s | 66 kB 00:00 2026-02-24T18:19:48.236 INFO:teuthology.orchestra.run.trial110.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-24T18:19:48.239 INFO:teuthology.orchestra.run.trial032.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.0 MB/s | 25 kB 00:00 2026-02-24T18:19:48.240 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-24T18:19:48.247 INFO:teuthology.orchestra.run.trial032.stdout:(27/35): libarrow-9.0.0-15.el9.x86_64.rpm 164 MB/s | 4.4 MB 00:00 2026-02-24T18:19:48.247 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-24T18:19:48.250 INFO:teuthology.orchestra.run.trial110.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-24T18:19:48.251 INFO:teuthology.orchestra.run.trial032.stdout:(28/35): liboath-2.6.12-1.el9.x86_64.rpm 4.2 MB/s | 49 kB 00:00 2026-02-24T18:19:48.254 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-24T18:19:48.259 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-24T18:19:48.300 INFO:teuthology.orchestra.run.trial032.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2026-02-24T18:19:48.307 INFO:teuthology.orchestra.run.trial032.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 29 MB/s | 191 kB 00:00 2026-02-24T18:19:48.319 INFO:teuthology.orchestra.run.trial032.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2026-02-24T18:19:48.342 INFO:teuthology.orchestra.run.trial032.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 45 MB/s | 1.6 MB 00:00 2026-02-24T18:19:48.402 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:48.403 INFO:teuthology.orchestra.run.trial110.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:48.428 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:48.430 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-24T18:19:48.436 INFO:teuthology.orchestra.run.trial110.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-24T18:19:48.662 INFO:teuthology.orchestra.run.trial110.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-24T18:19:48.663 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:48.679 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:48.682 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-24T18:19:48.687 INFO:teuthology.orchestra.run.trial110.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-24T18:19:48.689 INFO:teuthology.orchestra.run.trial110.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-24T18:19:48.690 INFO:teuthology.orchestra.run.trial110.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:48.708 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:48.753 INFO:teuthology.orchestra.run.trial112.stdout:(35/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 2.0 MB/s | 22 MB 00:10 2026-02-24T18:19:48.756 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:19:48.757 INFO:teuthology.orchestra.run.trial112.stdout:Total 5.5 MB/s | 63 MB 00:11 2026-02-24T18:19:48.824 INFO:teuthology.orchestra.run.trial110.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-24T18:19:48.826 INFO:teuthology.orchestra.run.trial110.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:48.850 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:19:48.869 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:19:48.869 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:19:49.099 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:49.099 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:19:49.099 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:49.119 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:19:49.120 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:19:49.136 INFO:teuthology.orchestra.run.trial110.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-24T18:19:49.137 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-24T18:19:49.411 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:19:49.420 INFO:teuthology.orchestra.run.trial112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-24T18:19:49.422 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-24T18:19:49.434 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-24T18:19:49.518 INFO:teuthology.orchestra.run.trial032.stdout:(33/35): librados2-19.2.3-689.gf90f4eae.el9.x86 2.9 MB/s | 3.4 MB 00:01 2026-02-24T18:19:49.553 INFO:teuthology.orchestra.run.trial032.stdout:(34/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 2.6 MB/s | 3.2 MB 00:01 2026-02-24T18:19:49.584 INFO:teuthology.orchestra.run.trial112.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-24T18:19:49.589 INFO:teuthology.orchestra.run.trial112.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:49.599 INFO:teuthology.orchestra.run.trial032.stdout:(35/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 3.1 MB/s | 22 MB 00:06 2026-02-24T18:19:49.603 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:19:49.603 INFO:teuthology.orchestra.run.trial032.stdout:Total 8.2 MB/s | 63 MB 00:07 2026-02-24T18:19:49.618 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:49.662 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:49.662 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:49.673 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:49.678 INFO:teuthology.orchestra.run.trial112.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-24T18:19:49.687 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-24T18:19:49.696 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:19:49.709 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:49.710 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:49.714 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:19:49.714 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:19:49.724 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:49.761 INFO:teuthology.orchestra.run.trial112.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-24T18:19:49.765 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-24T18:19:49.771 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-24T18:19:49.773 INFO:teuthology.orchestra.run.trial112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-24T18:19:49.776 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-24T18:19:49.781 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-24T18:19:49.920 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:49.921 INFO:teuthology.orchestra.run.trial112.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:49.946 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:49.948 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-24T18:19:49.955 INFO:teuthology.orchestra.run.trial112.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-24T18:19:49.964 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:19:49.965 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:19:50.120 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:50.122 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:50.182 INFO:teuthology.orchestra.run.trial112.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-24T18:19:50.183 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:50.199 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:50.203 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-24T18:19:50.207 INFO:teuthology.orchestra.run.trial112.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-24T18:19:50.209 INFO:teuthology.orchestra.run.trial112.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-24T18:19:50.210 INFO:teuthology.orchestra.run.trial112.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:50.230 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:50.259 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:19:50.267 INFO:teuthology.orchestra.run.trial032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-24T18:19:50.270 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-24T18:19:50.282 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-24T18:19:50.344 INFO:teuthology.orchestra.run.trial112.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-24T18:19:50.346 INFO:teuthology.orchestra.run.trial112.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:50.405 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:50.408 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:50.436 INFO:teuthology.orchestra.run.trial032.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-24T18:19:50.440 INFO:teuthology.orchestra.run.trial032.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:50.447 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:50.447 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T18:19:50.447 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T18:19:50.448 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:50.451 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:50.467 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:50.481 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:50.481 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /sys 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /proc 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /mnt 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /var/tmp 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /home 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /root 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout:skipping the directory /tmp 2026-02-24T18:19:50.482 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:50.510 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:50.511 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:50.521 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:50.525 INFO:teuthology.orchestra.run.trial032.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-24T18:19:50.534 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-24T18:19:50.557 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:50.557 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:50.570 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:50.607 INFO:teuthology.orchestra.run.trial032.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-24T18:19:50.611 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-24T18:19:50.616 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:50.616 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:19:50.616 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:50.617 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-24T18:19:50.620 INFO:teuthology.orchestra.run.trial032.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-24T18:19:50.623 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-24T18:19:50.628 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-24T18:19:50.652 INFO:teuthology.orchestra.run.trial112.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-24T18:19:50.654 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-24T18:19:50.772 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:50.773 INFO:teuthology.orchestra.run.trial032.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:50.800 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:19:50.803 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-24T18:19:50.810 INFO:teuthology.orchestra.run.trial032.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-24T18:19:50.932 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:50.956 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:50.958 INFO:teuthology.orchestra.run.trial018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:50.972 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:50.972 INFO:teuthology.orchestra.run.trial018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:51.028 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:51.028 INFO:teuthology.orchestra.run.trial018.stdout:Creating group 'qat' with GID 992. 2026-02-24T18:19:51.028 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:51.041 INFO:teuthology.orchestra.run.trial032.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-24T18:19:51.042 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:51.058 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:19:51.062 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-24T18:19:51.067 INFO:teuthology.orchestra.run.trial032.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-24T18:19:51.069 INFO:teuthology.orchestra.run.trial032.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-24T18:19:51.070 INFO:teuthology.orchestra.run.trial032.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:51.086 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:19:51.202 INFO:teuthology.orchestra.run.trial032.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-24T18:19:51.204 INFO:teuthology.orchestra.run.trial032.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:51.316 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-24T18:19:51.316 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-24T18:19:51.316 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-24T18:19:51.317 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-24T18:19:51.318 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-24T18:19:51.319 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-24T18:19:51.381 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:51.381 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout:Upgraded: 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.382 INFO:teuthology.orchestra.run.trial018.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T18:19:51.383 INFO:teuthology.orchestra.run.trial018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:51.384 INFO:teuthology.orchestra.run.trial018.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:51.385 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:19:51.426 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-test 2026-02-24T18:19:51.483 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:19:51.483 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:19:51.483 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:51.519 INFO:teuthology.orchestra.run.trial032.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-24T18:19:51.520 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-24T18:19:51.638 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:51.641 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:51.747 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:16 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:19:51.832 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repository Size 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout:Installing dependencies: 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-24T18:19:51.833 INFO:teuthology.orchestra.run.trial018.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout:Install 5 Packages 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 50 M 2026-02-24T18:19:51.834 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 212 M 2026-02-24T18:19:51.835 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:19:51.922 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:51.925 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:51.966 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:51.967 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T18:19:51.967 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T18:19:51.967 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:51.970 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:52.047 INFO:teuthology.orchestra.run.trial018.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 1.5 MB/s | 233 kB 00:00 2026-02-24T18:19:52.100 INFO:teuthology.orchestra.run.trial018.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 5.6 MB/s | 303 kB 00:00 2026-02-24T18:19:52.126 INFO:teuthology.orchestra.run.trial018.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 64 kB 00:00 2026-02-24T18:19:52.234 INFO:teuthology.orchestra.run.trial018.stdout:(4/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 480 kB/s | 163 kB 00:00 2026-02-24T18:19:52.521 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:52.524 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:52.807 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:19:52.810 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:52.849 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:19:52.849 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T18:19:52.849 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T18:19:52.850 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:52.852 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:55.014 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:55.014 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /sys 2026-02-24T18:19:55.014 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /proc 2026-02-24T18:19:55.014 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /mnt 2026-02-24T18:19:55.015 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /var/tmp 2026-02-24T18:19:55.015 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /home 2026-02-24T18:19:55.015 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /root 2026-02-24T18:19:55.015 INFO:teuthology.orchestra.run.trial110.stdout:skipping the directory /tmp 2026-02-24T18:19:55.015 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:55.366 INFO:teuthology.orchestra.run.trial018.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 14 MB/s | 50 MB 00:03 2026-02-24T18:19:55.367 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:19:55.367 INFO:teuthology.orchestra.run.trial018.stdout:Total 14 MB/s | 50 MB 00:03 2026-02-24T18:19:55.395 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:19:55.403 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:19:55.404 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:19:55.466 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:55.490 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:55.492 INFO:teuthology.orchestra.run.trial110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:55.505 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:55.505 INFO:teuthology.orchestra.run.trial110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:55.562 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:55.562 INFO:teuthology.orchestra.run.trial110.stdout:Creating group 'qat' with GID 992. 2026-02-24T18:19:55.562 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:55.591 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:19:55.591 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:19:55.806 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:19:55.812 INFO:teuthology.orchestra.run.trial018.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-24T18:19:55.817 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-24T18:19:55.825 INFO:teuthology.orchestra.run.trial018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-24T18:19:55.826 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:19:55.844 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-24T18:19:55.844 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-24T18:19:55.845 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-24T18:19:55.846 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-24T18:19:55.847 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-24T18:19:55.911 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:55.911 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout:Upgraded: 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.912 INFO:teuthology.orchestra.run.trial110.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.913 INFO:teuthology.orchestra.run.trial110.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:55.914 INFO:teuthology.orchestra.run.trial110.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:55.915 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:19:55.954 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-test 2026-02-24T18:19:56.267 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:19 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:19:56.351 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:19:56.352 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:56.352 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repository Size 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout:Installing dependencies: 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:56.353 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout:Install 5 Packages 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 50 M 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 212 M 2026-02-24T18:19:56.354 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:19:56.486 INFO:teuthology.orchestra.run.trial110.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 3.6 MB/s | 233 kB 00:00 2026-02-24T18:19:56.505 INFO:teuthology.orchestra.run.trial110.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 16 MB/s | 303 kB 00:00 2026-02-24T18:19:56.514 INFO:teuthology.orchestra.run.trial110.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.5 MB/s | 64 kB 00:00 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /sys 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /proc 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /mnt 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /var/tmp 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /home 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /root 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout:skipping the directory /tmp 2026-02-24T18:19:56.530 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:56.693 INFO:teuthology.orchestra.run.trial110.stdout:(4/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 604 kB/s | 163 kB 00:00 2026-02-24T18:19:56.981 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:57.004 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:57.005 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:19:57.005 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T18:19:57.005 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:57.005 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:57.005 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.006 INFO:teuthology.orchestra.run.trial112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:57.019 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:57.019 INFO:teuthology.orchestra.run.trial112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:57.074 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:57.074 INFO:teuthology.orchestra.run.trial112.stdout:Creating group 'qat' with GID 992. 2026-02-24T18:19:57.074 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.353 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-24T18:19:57.353 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-24T18:19:57.353 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-24T18:19:57.353 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-24T18:19:57.354 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-24T18:19:57.355 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-24T18:19:57.356 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /sys 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /proc 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /mnt 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /var/tmp 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /home 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /root 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout:skipping the directory /tmp 2026-02-24T18:19:57.367 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout:Upgraded: 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T18:19:57.421 INFO:teuthology.orchestra.run.trial112.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.422 INFO:teuthology.orchestra.run.trial112.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.423 INFO:teuthology.orchestra.run.trial112.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.424 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:19:57.464 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-test 2026-02-24T18:19:57.780 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:21 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:19:57.825 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:19:57.848 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:57.850 INFO:teuthology.orchestra.run.trial032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:57.861 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:19:57.861 INFO:teuthology.orchestra.run.trial032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:57.864 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repository Size 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout:Installing dependencies: 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T18:19:57.865 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout:Install 5 Packages 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 50 M 2026-02-24T18:19:57.866 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 212 M 2026-02-24T18:19:57.867 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:19:57.915 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:57.916 INFO:teuthology.orchestra.run.trial032.stdout:Creating group 'qat' with GID 992. 2026-02-24T18:19:57.916 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:58.120 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:19:58.138 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:19:58.146 INFO:teuthology.orchestra.run.trial112.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 7.5 MB/s | 233 kB 00:00 2026-02-24T18:19:58.159 INFO:teuthology.orchestra.run.trial112.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 24 MB/s | 303 kB 00:00 2026-02-24T18:19:58.164 INFO:teuthology.orchestra.run.trial112.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 13 MB/s | 64 kB 00:00 2026-02-24T18:19:58.193 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-24T18:19:58.193 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-24T18:19:58.193 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-24T18:19:58.194 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-24T18:19:58.195 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-24T18:19:58.196 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-24T18:19:58.259 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:19:58.259 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:58.259 INFO:teuthology.orchestra.run.trial032.stdout:Upgraded: 2026-02-24T18:19:58.259 INFO:teuthology.orchestra.run.trial032.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.260 INFO:teuthology.orchestra.run.trial032.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T18:19:58.261 INFO:teuthology.orchestra.run.trial032.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T18:19:58.262 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:58.263 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:19:58.300 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-test 2026-02-24T18:19:58.353 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:19:58.353 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-24T18:19:58.354 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-24T18:19:58.354 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-24T18:19:58.354 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-24T18:19:58.413 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-24T18:19:58.413 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:58.413 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:19:58.413 INFO:teuthology.orchestra.run.trial018.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:58.414 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:19:58.448 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph 2026-02-24T18:19:58.589 INFO:teuthology.orchestra.run.trial112.stdout:(4/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 345 kB/s | 163 kB 00:00 2026-02-24T18:19:58.612 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:18 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:19:58.695 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repository Size 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:Installing dependencies: 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout:Install 5 Packages 2026-02-24T18:19:58.696 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:19:58.697 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 50 M 2026-02-24T18:19:58.697 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 212 M 2026-02-24T18:19:58.697 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:19:58.762 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:23 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:19:58.826 INFO:teuthology.orchestra.run.trial032.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 3.3 MB/s | 233 kB 00:00 2026-02-24T18:19:58.850 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout:====================================================================================== 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repository Size 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout:====================================================================================== 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout:Installing dependencies: 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-24T18:19:58.853 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T18:19:58.854 INFO:teuthology.orchestra.run.trial018.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T18:19:58.855 INFO:teuthology.orchestra.run.trial018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T18:19:58.856 INFO:teuthology.orchestra.run.trial018.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T18:19:58.857 INFO:teuthology.orchestra.run.trial018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T18:19:58.858 INFO:teuthology.orchestra.run.trial018.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T18:19:58.859 INFO:teuthology.orchestra.run.trial018.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T18:19:58.860 INFO:teuthology.orchestra.run.trial018.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout:Installing weak dependencies: 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T18:19:58.861 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:58.862 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:19:58.862 INFO:teuthology.orchestra.run.trial018.stdout:====================================================================================== 2026-02-24T18:19:58.862 INFO:teuthology.orchestra.run.trial018.stdout:Install 99 Packages 2026-02-24T18:19:58.862 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:19:58.863 INFO:teuthology.orchestra.run.trial032.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 8.3 MB/s | 303 kB 00:00 2026-02-24T18:19:58.863 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 95 M 2026-02-24T18:19:58.863 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 520 M 2026-02-24T18:19:58.863 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:19:58.893 INFO:teuthology.orchestra.run.trial032.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.0 MB/s | 64 kB 00:00 2026-02-24T18:19:59.500 INFO:teuthology.orchestra.run.trial018.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 24 kB/s | 6.5 kB 00:00 2026-02-24T18:20:00.031 INFO:teuthology.orchestra.run.trial032.stdout:(4/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 128 kB/s | 163 kB 00:01 2026-02-24T18:20:00.805 INFO:teuthology.orchestra.run.trial018.stdout:(2/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 3.6 MB/s | 4.7 MB 00:01 2026-02-24T18:20:02.538 INFO:teuthology.orchestra.run.trial032.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 13 MB/s | 50 MB 00:03 2026-02-24T18:20:02.541 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:02.541 INFO:teuthology.orchestra.run.trial032.stdout:Total 13 MB/s | 50 MB 00:03 2026-02-24T18:20:02.547 INFO:teuthology.orchestra.run.trial018.stdout:(3/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 332 kB/s | 1.1 MB 00:03 2026-02-24T18:20:02.568 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:02.576 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:02.577 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:02.596 INFO:teuthology.orchestra.run.trial018.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 638 kB/s | 31 kB 00:00 2026-02-24T18:20:02.764 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:02.764 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:02.790 INFO:teuthology.orchestra.run.trial018.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 775 kB/s | 150 kB 00:00 2026-02-24T18:20:02.978 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:02.984 INFO:teuthology.orchestra.run.trial032.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-24T18:20:02.988 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-24T18:20:02.996 INFO:teuthology.orchestra.run.trial032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-24T18:20:02.997 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:03.285 INFO:teuthology.orchestra.run.trial018.stdout:(6/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_6 6.9 MB/s | 17 MB 00:02 2026-02-24T18:20:03.574 INFO:teuthology.orchestra.run.trial018.stdout:(7/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 572 kB/s | 2.4 MB 00:04 2026-02-24T18:20:03.623 INFO:teuthology.orchestra.run.trial018.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 455 kB/s | 22 kB 00:00 2026-02-24T18:20:03.741 INFO:teuthology.orchestra.run.trial112.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 8.8 MB/s | 50 MB 00:05 2026-02-24T18:20:03.742 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:03.742 INFO:teuthology.orchestra.run.trial112.stdout:Total 8.6 MB/s | 50 MB 00:05 2026-02-24T18:20:03.769 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:03.778 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:03.779 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:03.830 INFO:teuthology.orchestra.run.trial018.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 1.2 MB/s | 253 kB 00:00 2026-02-24T18:20:03.869 INFO:teuthology.orchestra.run.trial018.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 1.2 MB/s | 49 kB 00:00 2026-02-24T18:20:03.909 INFO:teuthology.orchestra.run.trial018.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 426 kB/s | 17 kB 00:00 2026-02-24T18:20:03.965 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:03.965 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:04.044 INFO:teuthology.orchestra.run.trial018.stdout:(12/99): ceph-mgr-diskprediction-local-19.2.3-6 9.7 MB/s | 7.4 MB 00:00 2026-02-24T18:20:04.105 INFO:teuthology.orchestra.run.trial018.stdout:(13/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 1.5 MB/s | 299 kB 00:00 2026-02-24T18:20:04.123 INFO:teuthology.orchestra.run.trial018.stdout:(14/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 9.6 MB/s | 769 kB 00:00 2026-02-24T18:20:04.178 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:04.178 INFO:teuthology.orchestra.run.trial018.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 727 kB/s | 40 kB 00:00 2026-02-24T18:20:04.181 INFO:teuthology.orchestra.run.trial018.stdout:(16/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 4.5 MB/s | 351 kB 00:00 2026-02-24T18:20:04.183 INFO:teuthology.orchestra.run.trial112.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-24T18:20:04.187 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-24T18:20:04.195 INFO:teuthology.orchestra.run.trial112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-24T18:20:04.196 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:04.220 INFO:teuthology.orchestra.run.trial018.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 1.7 MB/s | 72 kB 00:00 2026-02-24T18:20:04.221 INFO:teuthology.orchestra.run.trial018.stdout:(18/99): libgfortran-11.5.0-14.el9.x86_64.rpm 19 MB/s | 794 kB 00:00 2026-02-24T18:20:04.254 INFO:teuthology.orchestra.run.trial018.stdout:(19/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 7.6 MB/s | 253 kB 00:00 2026-02-24T18:20:04.256 INFO:teuthology.orchestra.run.trial018.stdout:(20/99): libquadmath-11.5.0-14.el9.x86_64.rpm 5.0 MB/s | 184 kB 00:00 2026-02-24T18:20:04.288 INFO:teuthology.orchestra.run.trial018.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 3.2 MB/s | 106 kB 00:00 2026-02-24T18:20:04.296 INFO:teuthology.orchestra.run.trial018.stdout:(22/99): python3-cryptography-36.0.1-5.el9.x86_ 30 MB/s | 1.2 MB 00:00 2026-02-24T18:20:04.319 INFO:teuthology.orchestra.run.trial018.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 4.4 MB/s | 135 kB 00:00 2026-02-24T18:20:04.333 INFO:teuthology.orchestra.run.trial018.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 4.0 MB/s | 150 kB 00:00 2026-02-24T18:20:04.352 INFO:teuthology.orchestra.run.trial018.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 3.7 MB/s | 126 kB 00:00 2026-02-24T18:20:04.363 INFO:teuthology.orchestra.run.trial018.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 7.0 MB/s | 218 kB 00:00 2026-02-24T18:20:04.387 INFO:teuthology.orchestra.run.trial018.stdout:(27/99): unzip-6.0-59.el9.x86_64.rpm 5.1 MB/s | 182 kB 00:00 2026-02-24T18:20:04.393 INFO:teuthology.orchestra.run.trial018.stdout:(28/99): zip-3.0-35.el9.x86_64.rpm 8.6 MB/s | 266 kB 00:00 2026-02-24T18:20:05.207 INFO:teuthology.orchestra.run.trial018.stdout:(29/99): flexiblas-3.0.4-9.el9.x86_64.rpm 36 kB/s | 30 kB 00:00 2026-02-24T18:20:05.250 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:05.269 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:05.364 INFO:teuthology.orchestra.run.trial018.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 94 kB/s | 15 kB 00:00 2026-02-24T18:20:05.500 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:05.500 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-24T18:20:05.501 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-24T18:20:05.501 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-24T18:20:05.501 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-24T18:20:05.558 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:05.559 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:05.593 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph 2026-02-24T18:20:05.679 INFO:teuthology.orchestra.run.trial018.stdout:(31/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 784 kB/s | 246 kB 00:00 2026-02-24T18:20:05.759 INFO:teuthology.orchestra.run.trial018.stdout:(32/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 2.2 MB/s | 3.0 MB 00:01 2026-02-24T18:20:05.768 INFO:teuthology.orchestra.run.trial018.stdout:(33/99): ceph-mgr-dashboard-19.2.3-689.gf90f4ea 1.3 MB/s | 3.8 MB 00:02 2026-02-24T18:20:05.839 INFO:teuthology.orchestra.run.trial018.stdout:(34/99): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2026-02-24T18:20:05.912 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:25 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:05.917 INFO:teuthology.orchestra.run.trial018.stdout:(35/99): openblas-0.3.29-1.el9.x86_64.rpm 267 kB/s | 42 kB 00:00 2026-02-24T18:20:06.000 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout:====================================================================================== 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repository Size 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout:====================================================================================== 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout:Installing dependencies: 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-24T18:20:06.003 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T18:20:06.004 INFO:teuthology.orchestra.run.trial032.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T18:20:06.005 INFO:teuthology.orchestra.run.trial032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T18:20:06.006 INFO:teuthology.orchestra.run.trial032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T18:20:06.007 INFO:teuthology.orchestra.run.trial032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T18:20:06.008 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T18:20:06.009 INFO:teuthology.orchestra.run.trial032.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout:Installing weak dependencies: 2026-02-24T18:20:06.010 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout:====================================================================================== 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout:Install 99 Packages 2026-02-24T18:20:06.011 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:06.012 INFO:teuthology.orchestra.run.trial018.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2026-02-24T18:20:06.013 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 95 M 2026-02-24T18:20:06.013 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 520 M 2026-02-24T18:20:06.013 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:06.050 INFO:teuthology.orchestra.run.trial110.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 5.2 MB/s | 50 MB 00:09 2026-02-24T18:20:06.051 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:06.051 INFO:teuthology.orchestra.run.trial110.stdout:Total 5.2 MB/s | 50 MB 00:09 2026-02-24T18:20:06.077 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:06.086 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:06.086 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:06.163 INFO:teuthology.orchestra.run.trial018.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2026-02-24T18:20:06.250 INFO:teuthology.orchestra.run.trial018.stdout:(38/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 11 MB/s | 5.3 MB 00:00 2026-02-24T18:20:06.272 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:06.272 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:06.323 INFO:teuthology.orchestra.run.trial018.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 1.5 MB/s | 249 kB 00:00 2026-02-24T18:20:06.410 INFO:teuthology.orchestra.run.trial018.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 1.1 MB/s | 177 kB 00:00 2026-02-24T18:20:06.485 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:06.488 INFO:teuthology.orchestra.run.trial018.stdout:(41/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 7.4 MB/s | 1.2 MB 00:00 2026-02-24T18:20:06.493 INFO:teuthology.orchestra.run.trial110.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-24T18:20:06.498 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-24T18:20:06.506 INFO:teuthology.orchestra.run.trial110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-24T18:20:06.507 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:06.520 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:06.539 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:06.569 INFO:teuthology.orchestra.run.trial018.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2026-02-24T18:20:06.647 INFO:teuthology.orchestra.run.trial018.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 219 kB/s | 35 kB 00:00 2026-02-24T18:20:06.761 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:06.761 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-24T18:20:06.762 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-24T18:20:06.762 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-24T18:20:06.762 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-24T18:20:06.780 INFO:teuthology.orchestra.run.trial032.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 27 kB/s | 6.5 kB 00:00 2026-02-24T18:20:06.808 INFO:teuthology.orchestra.run.trial018.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 2.7 MB/s | 442 kB 00:00 2026-02-24T18:20:06.819 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:06.820 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:06.855 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph 2026-02-24T18:20:06.906 INFO:teuthology.orchestra.run.trial018.stdout:(45/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 18 MB/s | 6.1 MB 00:00 2026-02-24T18:20:06.968 INFO:teuthology.orchestra.run.trial018.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 480 kB/s | 77 kB 00:00 2026-02-24T18:20:07.068 INFO:teuthology.orchestra.run.trial018.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 1.6 MB/s | 267 kB 00:00 2026-02-24T18:20:07.128 INFO:teuthology.orchestra.run.trial018.stdout:(48/99): python3-pyasn1-0.4.8-7.el9.noarch.rpm 987 kB/s | 157 kB 00:00 2026-02-24T18:20:07.172 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:31 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:07.238 INFO:teuthology.orchestra.run.trial018.stdout:(49/99): python3-pyasn1-modules-0.4.8-7.el9.noa 1.6 MB/s | 277 kB 00:00 2026-02-24T18:20:07.260 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:07.262 INFO:teuthology.orchestra.run.trial112.stdout:====================================================================================== 2026-02-24T18:20:07.262 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repository Size 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout:====================================================================================== 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout:Installing dependencies: 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-24T18:20:07.263 INFO:teuthology.orchestra.run.trial112.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T18:20:07.264 INFO:teuthology.orchestra.run.trial112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T18:20:07.265 INFO:teuthology.orchestra.run.trial112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T18:20:07.266 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T18:20:07.267 INFO:teuthology.orchestra.run.trial112.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T18:20:07.268 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T18:20:07.269 INFO:teuthology.orchestra.run.trial112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout:Installing weak dependencies: 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-24T18:20:07.270 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout:====================================================================================== 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout:Install 99 Packages 2026-02-24T18:20:07.271 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:07.272 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 95 M 2026-02-24T18:20:07.272 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 520 M 2026-02-24T18:20:07.272 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:07.280 INFO:teuthology.orchestra.run.trial032.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 1.5 MB/s | 1.1 MB 00:00 2026-02-24T18:20:07.287 INFO:teuthology.orchestra.run.trial018.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 337 kB/s | 54 kB 00:00 2026-02-24T18:20:07.446 INFO:teuthology.orchestra.run.trial018.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 42 kB 00:00 2026-02-24T18:20:07.606 INFO:teuthology.orchestra.run.trial018.stdout:(52/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 1.2 MB/s | 189 kB 00:00 2026-02-24T18:20:07.669 INFO:teuthology.orchestra.run.trial032.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 2.1 MB/s | 2.4 MB 00:01 2026-02-24T18:20:07.725 INFO:teuthology.orchestra.run.trial032.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 561 kB/s | 31 kB 00:00 2026-02-24T18:20:07.764 INFO:teuthology.orchestra.run.trial018.stdout:(53/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 572 kB/s | 90 kB 00:00 2026-02-24T18:20:07.783 INFO:teuthology.orchestra.run.trial032.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 2.5 MB/s | 150 kB 00:00 2026-02-24T18:20:07.792 INFO:teuthology.orchestra.run.trial018.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 821 kB/s | 22 kB 00:00 2026-02-24T18:20:07.811 INFO:teuthology.orchestra.run.trial112.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 28 kB/s | 6.5 kB 00:00 2026-02-24T18:20:07.845 INFO:teuthology.orchestra.run.trial018.stdout:(55/99): protobuf-compiler-3.14.0-17.el9.x86_64 16 MB/s | 862 kB 00:00 2026-02-24T18:20:08.148 INFO:teuthology.orchestra.run.trial018.stdout:(56/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 21 MB/s | 19 MB 00:00 2026-02-24T18:20:08.216 INFO:teuthology.orchestra.run.trial018.stdout:(57/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.5 MB/s | 551 kB 00:00 2026-02-24T18:20:08.270 INFO:teuthology.orchestra.run.trial018.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 2.8 MB/s | 151 kB 00:00 2026-02-24T18:20:08.295 INFO:teuthology.orchestra.run.trial018.stdout:(59/99): python3-babel-2.9.1-2.el9.noarch.rpm 2.5 MB/s | 6.0 MB 00:02 2026-02-24T18:20:08.328 INFO:teuthology.orchestra.run.trial018.stdout:(60/99): grpc-data-1.46.7-10.el9.noarch.rpm 108 kB/s | 19 kB 00:00 2026-02-24T18:20:08.343 INFO:teuthology.orchestra.run.trial018.stdout:(61/99): python3-asyncssh-2.13.2-5.el9.noarch.r 7.3 MB/s | 548 kB 00:00 2026-02-24T18:20:08.386 INFO:teuthology.orchestra.run.trial018.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2026-02-24T18:20:08.416 INFO:teuthology.orchestra.run.trial112.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 1.3 MB/s | 1.1 MB 00:00 2026-02-24T18:20:08.427 INFO:teuthology.orchestra.run.trial018.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 780 kB/s | 32 kB 00:00 2026-02-24T18:20:08.434 INFO:teuthology.orchestra.run.trial032.stdout:(6/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 2.9 MB/s | 4.7 MB 00:01 2026-02-24T18:20:08.439 INFO:teuthology.orchestra.run.trial018.stdout:(64/99): python3-backports-tarfile-1.2.0-1.el9. 539 kB/s | 60 kB 00:00 2026-02-24T18:20:08.469 INFO:teuthology.orchestra.run.trial018.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 344 kB/s | 14 kB 00:00 2026-02-24T18:20:08.475 INFO:teuthology.orchestra.run.trial018.stdout:(66/99): python3-autocommand-2.2.2-8.el9.noarch 164 kB/s | 29 kB 00:00 2026-02-24T18:20:08.527 INFO:teuthology.orchestra.run.trial018.stdout:(67/99): python3-cherrypy-18.6.1-2.el9.noarch.r 6.0 MB/s | 358 kB 00:00 2026-02-24T18:20:08.563 INFO:teuthology.orchestra.run.trial018.stdout:(68/99): python3-cheroot-10.0.1-4.el9.noarch.rp 1.4 MB/s | 173 kB 00:00 2026-02-24T18:20:08.633 INFO:teuthology.orchestra.run.trial018.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 2.0 MB/s | 144 kB 00:00 2026-02-24T18:20:08.640 INFO:teuthology.orchestra.run.trial018.stdout:(70/99): python3-google-auth-2.45.0-1.el9.noarc 1.5 MB/s | 254 kB 00:00 2026-02-24T18:20:08.655 INFO:teuthology.orchestra.run.trial018.stdout:(71/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 16 MB/s | 2.0 MB 00:00 2026-02-24T18:20:08.686 INFO:teuthology.orchestra.run.trial018.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.2 MB/s | 56 kB 00:00 2026-02-24T18:20:08.696 INFO:teuthology.orchestra.run.trial018.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 260 kB/s | 11 kB 00:00 2026-02-24T18:20:08.700 INFO:teuthology.orchestra.run.trial018.stdout:(74/99): python3-influxdb-5.3.1-1.el9.noarch.rp 2.0 MB/s | 139 kB 00:00 2026-02-24T18:20:08.729 INFO:teuthology.orchestra.run.trial018.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 408 kB/s | 18 kB 00:00 2026-02-24T18:20:08.737 INFO:teuthology.orchestra.run.trial018.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 559 kB/s | 23 kB 00:00 2026-02-24T18:20:08.744 INFO:teuthology.orchestra.run.trial018.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 447 kB/s | 20 kB 00:00 2026-02-24T18:20:08.773 INFO:teuthology.orchestra.run.trial018.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 445 kB/s | 19 kB 00:00 2026-02-24T18:20:08.779 INFO:teuthology.orchestra.run.trial018.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 640 kB/s | 26 kB 00:00 2026-02-24T18:20:08.819 INFO:teuthology.orchestra.run.trial018.stdout:(80/99): python3-logutils-0.3.5-21.el9.noarch.r 1.0 MB/s | 46 kB 00:00 2026-02-24T18:20:08.821 INFO:teuthology.orchestra.run.trial018.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2026-02-24T18:20:08.829 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:20:08.847 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:08.862 INFO:teuthology.orchestra.run.trial018.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2026-02-24T18:20:08.869 INFO:teuthology.orchestra.run.trial018.stdout:(83/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.7 MB/s | 86 kB 00:00 2026-02-24T18:20:08.882 INFO:teuthology.orchestra.run.trial112.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 1.9 MB/s | 2.4 MB 00:01 2026-02-24T18:20:08.906 INFO:teuthology.orchestra.run.trial018.stdout:(84/99): python3-kubernetes-26.1.0-3.el9.noarch 6.3 MB/s | 1.0 MB 00:00 2026-02-24T18:20:08.908 INFO:teuthology.orchestra.run.trial018.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 5.8 MB/s | 272 kB 00:00 2026-02-24T18:20:08.911 INFO:teuthology.orchestra.run.trial018.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2026-02-24T18:20:08.944 INFO:teuthology.orchestra.run.trial112.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 502 kB/s | 31 kB 00:00 2026-02-24T18:20:08.949 INFO:teuthology.orchestra.run.trial018.stdout:(87/99): python3-repoze-lru-0.7-16.el9.noarch.r 750 kB/s | 31 kB 00:00 2026-02-24T18:20:08.953 INFO:teuthology.orchestra.run.trial018.stdout:(88/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.9 MB/s | 90 kB 00:00 2026-02-24T18:20:08.970 INFO:teuthology.orchestra.run.trial018.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2026-02-24T18:20:08.991 INFO:teuthology.orchestra.run.trial018.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2026-02-24T18:20:09.001 INFO:teuthology.orchestra.run.trial112.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 2.6 MB/s | 150 kB 00:00 2026-02-24T18:20:09.005 INFO:teuthology.orchestra.run.trial018.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 2.4 MB/s | 125 kB 00:00 2026-02-24T18:20:09.014 INFO:teuthology.orchestra.run.trial018.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 815 kB/s | 36 kB 00:00 2026-02-24T18:20:09.024 INFO:teuthology.orchestra.run.trial032.stdout:(7/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 3.1 MB/s | 3.8 MB 00:01 2026-02-24T18:20:09.033 INFO:teuthology.orchestra.run.trial018.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 2.0 MB/s | 86 kB 00:00 2026-02-24T18:20:09.060 INFO:teuthology.orchestra.run.trial018.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 4.1 MB/s | 230 kB 00:00 2026-02-24T18:20:09.064 INFO:teuthology.orchestra.run.trial018.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2026-02-24T18:20:09.079 INFO:teuthology.orchestra.run.trial032.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 399 kB/s | 22 kB 00:00 2026-02-24T18:20:09.080 INFO:teuthology.orchestra.run.trial018.stdout:(96/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 9.0 MB/s | 427 kB 00:00 2026-02-24T18:20:09.101 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:20:09.102 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-24T18:20:09.102 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-24T18:20:09.102 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-24T18:20:09.102 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-24T18:20:09.106 INFO:teuthology.orchestra.run.trial018.stdout:(97/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.0 MB/s | 48 kB 00:00 2026-02-24T18:20:09.107 INFO:teuthology.orchestra.run.trial018.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 517 kB/s | 22 kB 00:00 2026-02-24T18:20:09.122 INFO:teuthology.orchestra.run.trial018.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2026-02-24T18:20:09.126 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:09.126 INFO:teuthology.orchestra.run.trial018.stdout:Total 9.3 MB/s | 95 MB 00:10 2026-02-24T18:20:09.160 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-24T18:20:09.160 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:09.161 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:09.196 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph 2026-02-24T18:20:09.198 INFO:teuthology.orchestra.run.trial032.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 2.1 MB/s | 253 kB 00:00 2026-02-24T18:20:09.258 INFO:teuthology.orchestra.run.trial032.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 825 kB/s | 49 kB 00:00 2026-02-24T18:20:09.318 INFO:teuthology.orchestra.run.trial032.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 282 kB/s | 17 kB 00:00 2026-02-24T18:20:09.443 INFO:teuthology.orchestra.run.trial032.stdout:(12/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 2.3 MB/s | 299 kB 00:00 2026-02-24T18:20:09.510 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:32 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:09.529 INFO:teuthology.orchestra.run.trial112.stdout:(6/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 2.8 MB/s | 4.7 MB 00:01 2026-02-24T18:20:09.571 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:09.596 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:09.599 INFO:teuthology.orchestra.run.trial110.stdout:====================================================================================== 2026-02-24T18:20:09.599 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repository Size 2026-02-24T18:20:09.599 INFO:teuthology.orchestra.run.trial110.stdout:====================================================================================== 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout:Installing dependencies: 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T18:20:09.600 INFO:teuthology.orchestra.run.trial110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T18:20:09.601 INFO:teuthology.orchestra.run.trial110.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout:Installing weak dependencies: 2026-02-24T18:20:09.602 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:====================================================================================== 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:Install 99 Packages 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 95 M 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 520 M 2026-02-24T18:20:09.603 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:09.608 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:09.609 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:09.743 INFO:teuthology.orchestra.run.trial112.stdout:(7/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 5.1 MB/s | 3.8 MB 00:00 2026-02-24T18:20:09.773 INFO:teuthology.orchestra.run.trial032.stdout:(13/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 2.3 MB/s | 769 kB 00:00 2026-02-24T18:20:09.806 INFO:teuthology.orchestra.run.trial112.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 355 kB/s | 22 kB 00:00 2026-02-24T18:20:09.828 INFO:teuthology.orchestra.run.trial032.stdout:(14/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 6.2 MB/s | 351 kB 00:00 2026-02-24T18:20:09.838 INFO:teuthology.orchestra.run.trial032.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 4.3 MB/s | 40 kB 00:00 2026-02-24T18:20:09.847 INFO:teuthology.orchestra.run.trial032.stdout:(16/99): libconfig-1.7.2-9.el9.x86_64.rpm 7.5 MB/s | 72 kB 00:00 2026-02-24T18:20:09.868 INFO:teuthology.orchestra.run.trial032.stdout:(17/99): libgfortran-11.5.0-14.el9.x86_64.rpm 38 MB/s | 794 kB 00:00 2026-02-24T18:20:09.869 INFO:teuthology.orchestra.run.trial112.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 3.9 MB/s | 253 kB 00:00 2026-02-24T18:20:09.880 INFO:teuthology.orchestra.run.trial032.stdout:(18/99): libquadmath-11.5.0-14.el9.x86_64.rpm 15 MB/s | 184 kB 00:00 2026-02-24T18:20:09.892 INFO:teuthology.orchestra.run.trial032.stdout:(19/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 21 MB/s | 253 kB 00:00 2026-02-24T18:20:09.926 INFO:teuthology.orchestra.run.trial032.stdout:(20/99): python3-cryptography-36.0.1-5.el9.x86_ 36 MB/s | 1.2 MB 00:00 2026-02-24T18:20:09.930 INFO:teuthology.orchestra.run.trial112.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 799 kB/s | 49 kB 00:00 2026-02-24T18:20:09.937 INFO:teuthology.orchestra.run.trial032.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 9.9 MB/s | 106 kB 00:00 2026-02-24T18:20:09.948 INFO:teuthology.orchestra.run.trial032.stdout:(22/99): python3-pycparser-2.20-6.el9.noarch.rp 12 MB/s | 135 kB 00:00 2026-02-24T18:20:09.959 INFO:teuthology.orchestra.run.trial032.stdout:(23/99): python3-pyparsing-2.4.7-9.el9.noarch.r 13 MB/s | 150 kB 00:00 2026-02-24T18:20:09.971 INFO:teuthology.orchestra.run.trial032.stdout:(24/99): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-24T18:20:09.987 INFO:teuthology.orchestra.run.trial032.stdout:(25/99): python3-urllib3-1.26.5-7.el9.noarch.rp 13 MB/s | 218 kB 00:00 2026-02-24T18:20:09.992 INFO:teuthology.orchestra.run.trial112.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 272 kB/s | 17 kB 00:00 2026-02-24T18:20:09.998 INFO:teuthology.orchestra.run.trial032.stdout:(26/99): unzip-6.0-59.el9.x86_64.rpm 15 MB/s | 182 kB 00:00 2026-02-24T18:20:10.009 INFO:teuthology.orchestra.run.trial032.stdout:(27/99): zip-3.0-35.el9.x86_64.rpm 24 MB/s | 266 kB 00:00 2026-02-24T18:20:10.026 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:10.026 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:10.062 INFO:teuthology.orchestra.run.trial112.stdout:(12/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 4.2 MB/s | 299 kB 00:00 2026-02-24T18:20:10.084 INFO:teuthology.orchestra.run.trial032.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 399 kB/s | 30 kB 00:00 2026-02-24T18:20:10.196 INFO:teuthology.orchestra.run.trial110.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 24 kB/s | 6.5 kB 00:00 2026-02-24T18:20:10.265 INFO:teuthology.orchestra.run.trial112.stdout:(13/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 3.7 MB/s | 769 kB 00:00 2026-02-24T18:20:10.316 INFO:teuthology.orchestra.run.trial032.stdout:(29/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 13 MB/s | 3.0 MB 00:00 2026-02-24T18:20:10.322 INFO:teuthology.orchestra.run.trial112.stdout:(14/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 6.1 MB/s | 351 kB 00:00 2026-02-24T18:20:10.331 INFO:teuthology.orchestra.run.trial112.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 4.2 MB/s | 40 kB 00:00 2026-02-24T18:20:10.341 INFO:teuthology.orchestra.run.trial112.stdout:(16/99): libconfig-1.7.2-9.el9.x86_64.rpm 7.5 MB/s | 72 kB 00:00 2026-02-24T18:20:10.363 INFO:teuthology.orchestra.run.trial112.stdout:(17/99): libgfortran-11.5.0-14.el9.x86_64.rpm 36 MB/s | 794 kB 00:00 2026-02-24T18:20:10.375 INFO:teuthology.orchestra.run.trial112.stdout:(18/99): libquadmath-11.5.0-14.el9.x86_64.rpm 15 MB/s | 184 kB 00:00 2026-02-24T18:20:10.379 INFO:teuthology.orchestra.run.trial032.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 235 kB/s | 15 kB 00:00 2026-02-24T18:20:10.387 INFO:teuthology.orchestra.run.trial112.stdout:(19/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 21 MB/s | 253 kB 00:00 2026-02-24T18:20:10.421 INFO:teuthology.orchestra.run.trial112.stdout:(20/99): python3-cryptography-36.0.1-5.el9.x86_ 37 MB/s | 1.2 MB 00:00 2026-02-24T18:20:10.431 INFO:teuthology.orchestra.run.trial112.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 9.8 MB/s | 106 kB 00:00 2026-02-24T18:20:10.442 INFO:teuthology.orchestra.run.trial112.stdout:(22/99): python3-pycparser-2.20-6.el9.noarch.rp 12 MB/s | 135 kB 00:00 2026-02-24T18:20:10.453 INFO:teuthology.orchestra.run.trial112.stdout:(23/99): python3-pyparsing-2.4.7-9.el9.noarch.r 13 MB/s | 150 kB 00:00 2026-02-24T18:20:10.465 INFO:teuthology.orchestra.run.trial112.stdout:(24/99): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-24T18:20:10.477 INFO:teuthology.orchestra.run.trial112.stdout:(25/99): python3-urllib3-1.26.5-7.el9.noarch.rp 18 MB/s | 218 kB 00:00 2026-02-24T18:20:10.479 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:10.481 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-24T18:20:10.488 INFO:teuthology.orchestra.run.trial112.stdout:(26/99): unzip-6.0-59.el9.x86_64.rpm 15 MB/s | 182 kB 00:00 2026-02-24T18:20:10.499 INFO:teuthology.orchestra.run.trial112.stdout:(27/99): zip-3.0-35.el9.x86_64.rpm 24 MB/s | 266 kB 00:00 2026-02-24T18:20:10.505 INFO:teuthology.orchestra.run.trial032.stdout:(31/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 1.9 MB/s | 246 kB 00:00 2026-02-24T18:20:10.512 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-24T18:20:10.518 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-24T18:20:10.539 INFO:teuthology.orchestra.run.trial018.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-24T18:20:10.545 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 5/99 2026-02-24T18:20:10.562 INFO:teuthology.orchestra.run.trial110.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 1.7 MB/s | 1.1 MB 00:00 2026-02-24T18:20:10.577 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-24T18:20:10.580 INFO:teuthology.orchestra.run.trial112.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 368 kB/s | 30 kB 00:00 2026-02-24T18:20:10.587 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-24T18:20:10.592 INFO:teuthology.orchestra.run.trial018.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-24T18:20:10.594 INFO:teuthology.orchestra.run.trial018.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-24T18:20:10.598 INFO:teuthology.orchestra.run.trial018.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-24T18:20:10.611 INFO:teuthology.orchestra.run.trial032.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2026-02-24T18:20:10.621 INFO:teuthology.orchestra.run.trial018.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-24T18:20:10.633 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-24T18:20:10.637 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-24T18:20:10.643 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-24T18:20:10.645 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-24T18:20:10.657 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-24T18:20:10.662 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-24T18:20:10.686 INFO:teuthology.orchestra.run.trial018.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-24T18:20:10.689 INFO:teuthology.orchestra.run.trial018.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-24T18:20:10.695 INFO:teuthology.orchestra.run.trial032.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 503 kB/s | 42 kB 00:00 2026-02-24T18:20:10.696 INFO:teuthology.orchestra.run.trial018.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-24T18:20:10.720 INFO:teuthology.orchestra.run.trial018.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-24T18:20:10.768 INFO:teuthology.orchestra.run.trial112.stdout:(29/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 16 MB/s | 3.0 MB 00:00 2026-02-24T18:20:10.771 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-24T18:20:10.783 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 23/99 2026-02-24T18:20:10.786 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-24T18:20:10.790 INFO:teuthology.orchestra.run.trial018.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-24T18:20:10.793 INFO:teuthology.orchestra.run.trial112.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 592 kB/s | 15 kB 00:00 2026-02-24T18:20:10.796 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-24T18:20:10.800 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-24T18:20:10.803 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-24T18:20:10.818 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-24T18:20:10.820 INFO:teuthology.orchestra.run.trial112.stdout:(31/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 8.9 MB/s | 246 kB 00:00 2026-02-24T18:20:10.838 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-24T18:20:10.843 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-24T18:20:10.847 INFO:teuthology.orchestra.run.trial112.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 6.8 MB/s | 188 kB 00:00 2026-02-24T18:20:10.848 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-24T18:20:10.859 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-24T18:20:10.868 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-24T18:20:10.872 INFO:teuthology.orchestra.run.trial112.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 1.6 MB/s | 42 kB 00:00 2026-02-24T18:20:10.874 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-24T18:20:10.893 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-24T18:20:10.902 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-24T18:20:10.938 INFO:teuthology.orchestra.run.trial112.stdout:(34/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 81 MB/s | 5.3 MB 00:00 2026-02-24T18:20:10.955 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-24T18:20:10.967 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-24T18:20:10.968 INFO:teuthology.orchestra.run.trial112.stdout:(35/99): protobuf-3.14.0-17.el9.x86_64.rpm 33 MB/s | 1.0 MB 00:00 2026-02-24T18:20:10.981 INFO:teuthology.orchestra.run.trial032.stdout:(34/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 19 MB/s | 5.3 MB 00:00 2026-02-24T18:20:10.983 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-24T18:20:10.987 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-24T18:20:10.995 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-24T18:20:11.035 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-24T18:20:11.036 INFO:teuthology.orchestra.run.trial110.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 2.2 MB/s | 2.4 MB 00:01 2026-02-24T18:20:11.057 INFO:teuthology.orchestra.run.trial112.stdout:(36/99): python3-babel-2.9.1-2.el9.noarch.rpm 67 MB/s | 6.0 MB 00:00 2026-02-24T18:20:11.086 INFO:teuthology.orchestra.run.trial112.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 8.4 MB/s | 244 kB 00:00 2026-02-24T18:20:11.098 INFO:teuthology.orchestra.run.trial110.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 505 kB/s | 31 kB 00:00 2026-02-24T18:20:11.114 INFO:teuthology.orchestra.run.trial112.stdout:(38/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 8.6 MB/s | 249 kB 00:00 2026-02-24T18:20:11.142 INFO:teuthology.orchestra.run.trial112.stdout:(39/99): python3-libstoragemgmt-1.10.1-1.el9.x8 6.2 MB/s | 177 kB 00:00 2026-02-24T18:20:11.150 INFO:teuthology.orchestra.run.trial032.stdout:(35/99): protobuf-3.14.0-17.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2026-02-24T18:20:11.163 INFO:teuthology.orchestra.run.trial110.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 2.3 MB/s | 150 kB 00:00 2026-02-24T18:20:11.173 INFO:teuthology.orchestra.run.trial112.stdout:(40/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 39 MB/s | 1.2 MB 00:00 2026-02-24T18:20:11.201 INFO:teuthology.orchestra.run.trial112.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.0 MB/s | 172 kB 00:00 2026-02-24T18:20:11.238 INFO:teuthology.orchestra.run.trial112.stdout:(42/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 6.1 MB/s | 17 MB 00:02 2026-02-24T18:20:11.238 INFO:teuthology.orchestra.run.trial112.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 940 kB/s | 35 kB 00:00 2026-02-24T18:20:11.266 INFO:teuthology.orchestra.run.trial112.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 15 MB/s | 442 kB 00:00 2026-02-24T18:20:11.293 INFO:teuthology.orchestra.run.trial112.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 2.9 MB/s | 77 kB 00:00 2026-02-24T18:20:11.298 INFO:teuthology.orchestra.run.trial032.stdout:(36/99): ceph-mgr-diskprediction-local-19.2.3-6 2.6 MB/s | 7.4 MB 00:02 2026-02-24T18:20:11.321 INFO:teuthology.orchestra.run.trial112.stdout:(46/99): python3-protobuf-3.14.0-17.el9.noarch. 9.4 MB/s | 267 kB 00:00 2026-02-24T18:20:11.349 INFO:teuthology.orchestra.run.trial112.stdout:(47/99): python3-pyasn1-0.4.8-7.el9.noarch.rpm 5.5 MB/s | 157 kB 00:00 2026-02-24T18:20:11.357 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-24T18:20:11.366 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-24T18:20:11.370 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-24T18:20:11.376 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-24T18:20:11.377 INFO:teuthology.orchestra.run.trial112.stdout:(48/99): python3-pyasn1-modules-0.4.8-7.el9.noa 9.6 MB/s | 277 kB 00:00 2026-02-24T18:20:11.378 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-24T18:20:11.403 INFO:teuthology.orchestra.run.trial112.stdout:(49/99): python3-requests-oauthlib-1.3.0-12.el9 2.0 MB/s | 54 kB 00:00 2026-02-24T18:20:11.405 INFO:teuthology.orchestra.run.trial018.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-24T18:20:11.425 INFO:teuthology.orchestra.run.trial032.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2026-02-24T18:20:11.448 INFO:teuthology.orchestra.run.trial018.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-24T18:20:11.454 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-24T18:20:11.456 INFO:teuthology.orchestra.run.trial032.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2026-02-24T18:20:11.458 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-24T18:20:11.463 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-24T18:20:11.467 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-24T18:20:11.474 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-24T18:20:11.478 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-24T18:20:11.505 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-24T18:20:11.515 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-24T18:20:11.517 INFO:teuthology.orchestra.run.trial112.stdout:(50/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 22 MB/s | 6.1 MB 00:00 2026-02-24T18:20:11.544 INFO:teuthology.orchestra.run.trial112.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.6 MB/s | 42 kB 00:00 2026-02-24T18:20:11.549 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-24T18:20:11.551 INFO:teuthology.orchestra.run.trial032.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 1.9 MB/s | 249 kB 00:00 2026-02-24T18:20:11.562 INFO:teuthology.orchestra.run.trial032.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 1.6 MB/s | 177 kB 00:00 2026-02-24T18:20:11.580 INFO:teuthology.orchestra.run.trial112.stdout:(52/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 109 MB/s | 19 MB 00:00 2026-02-24T18:20:11.581 INFO:teuthology.orchestra.run.trial112.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 4.9 MB/s | 189 kB 00:00 2026-02-24T18:20:11.607 INFO:teuthology.orchestra.run.trial112.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.4 MB/s | 90 kB 00:00 2026-02-24T18:20:11.669 INFO:teuthology.orchestra.run.trial032.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2026-02-24T18:20:11.725 INFO:teuthology.orchestra.run.trial032.stdout:(42/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 7.1 MB/s | 1.2 MB 00:00 2026-02-24T18:20:11.733 INFO:teuthology.orchestra.run.trial032.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 541 kB/s | 35 kB 00:00 2026-02-24T18:20:11.773 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-24T18:20:11.796 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-24T18:20:11.800 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-24T18:20:11.827 INFO:teuthology.orchestra.run.trial112.stdout:(55/99): lua-devel-5.4.4-4.el9.x86_64.rpm 91 kB/s | 22 kB 00:00 2026-02-24T18:20:11.838 INFO:teuthology.orchestra.run.trial110.stdout:(6/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 2.9 MB/s | 4.7 MB 00:01 2026-02-24T18:20:11.858 INFO:teuthology.orchestra.run.trial018.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-24T18:20:11.860 INFO:teuthology.orchestra.run.trial018.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-24T18:20:11.881 INFO:teuthology.orchestra.run.trial018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-24T18:20:11.881 INFO:teuthology.orchestra.run.trial032.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 2.9 MB/s | 442 kB 00:00 2026-02-24T18:20:11.966 INFO:teuthology.orchestra.run.trial032.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 907 kB/s | 77 kB 00:00 2026-02-24T18:20:12.049 INFO:teuthology.orchestra.run.trial032.stdout:(46/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 19 MB/s | 6.1 MB 00:00 2026-02-24T18:20:12.093 INFO:teuthology.orchestra.run.trial032.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 2.1 MB/s | 267 kB 00:00 2026-02-24T18:20:12.131 INFO:teuthology.orchestra.run.trial110.stdout:(7/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 3.9 MB/s | 3.8 MB 00:00 2026-02-24T18:20:12.156 INFO:teuthology.orchestra.run.trial032.stdout:(48/99): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.4 MB/s | 157 kB 00:00 2026-02-24T18:20:12.158 INFO:teuthology.orchestra.run.trial112.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.6 MB/s | 551 kB 00:00 2026-02-24T18:20:12.186 INFO:teuthology.orchestra.run.trial110.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 407 kB/s | 22 kB 00:00 2026-02-24T18:20:12.196 INFO:teuthology.orchestra.run.trial112.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 512 kB/s | 19 kB 00:00 2026-02-24T18:20:12.203 INFO:teuthology.orchestra.run.trial018.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-24T18:20:12.221 INFO:teuthology.orchestra.run.trial032.stdout:(49/99): python3-pyasn1-modules-0.4.8-7.el9.noa 2.1 MB/s | 277 kB 00:00 2026-02-24T18:20:12.241 INFO:teuthology.orchestra.run.trial032.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 633 kB/s | 54 kB 00:00 2026-02-24T18:20:12.242 INFO:teuthology.orchestra.run.trial112.stdout:(58/99): protobuf-compiler-3.14.0-17.el9.x86_64 1.3 MB/s | 862 kB 00:00 2026-02-24T18:20:12.252 INFO:teuthology.orchestra.run.trial112.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2026-02-24T18:20:12.275 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-24T18:20:12.298 INFO:teuthology.orchestra.run.trial110.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 2.2 MB/s | 253 kB 00:00 2026-02-24T18:20:12.302 INFO:teuthology.orchestra.run.trial112.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 588 kB/s | 29 kB 00:00 2026-02-24T18:20:12.325 INFO:teuthology.orchestra.run.trial032.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 494 kB/s | 42 kB 00:00 2026-02-24T18:20:12.350 INFO:teuthology.orchestra.run.trial112.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 1.2 MB/s | 60 kB 00:00 2026-02-24T18:20:12.354 INFO:teuthology.orchestra.run.trial110.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 878 kB/s | 49 kB 00:00 2026-02-24T18:20:12.399 INFO:teuthology.orchestra.run.trial112.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 903 kB/s | 43 kB 00:00 2026-02-24T18:20:12.411 INFO:teuthology.orchestra.run.trial110.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 299 kB/s | 17 kB 00:00 2026-02-24T18:20:12.428 INFO:teuthology.orchestra.run.trial112.stdout:(63/99): ceph-mgr-diskprediction-local-19.2.3-6 2.6 MB/s | 7.4 MB 00:02 2026-02-24T18:20:12.431 INFO:teuthology.orchestra.run.trial032.stdout:(52/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 1.7 MB/s | 189 kB 00:00 2026-02-24T18:20:12.437 INFO:teuthology.orchestra.run.trial112.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 848 kB/s | 32 kB 00:00 2026-02-24T18:20:12.488 INFO:teuthology.orchestra.run.trial112.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 3.3 MB/s | 173 kB 00:00 2026-02-24T18:20:12.516 INFO:teuthology.orchestra.run.trial032.stdout:(53/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.0 MB/s | 90 kB 00:00 2026-02-24T18:20:12.528 INFO:teuthology.orchestra.run.trial110.stdout:(12/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 2.5 MB/s | 299 kB 00:00 2026-02-24T18:20:12.548 INFO:teuthology.orchestra.run.trial112.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 5.8 MB/s | 358 kB 00:00 2026-02-24T18:20:12.553 INFO:teuthology.orchestra.run.trial112.stdout:(67/99): python3-certifi-2023.05.07-4.el9.noarc 113 kB/s | 14 kB 00:00 2026-02-24T18:20:12.556 INFO:teuthology.orchestra.run.trial112.stdout:(68/99): python3-asyncssh-2.13.2-5.el9.noarch.r 1.7 MB/s | 548 kB 00:00 2026-02-24T18:20:12.591 INFO:teuthology.orchestra.run.trial032.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2026-02-24T18:20:12.597 INFO:teuthology.orchestra.run.trial112.stdout:(69/99): python3-google-auth-2.45.0-1.el9.noarc 5.1 MB/s | 254 kB 00:00 2026-02-24T18:20:12.604 INFO:teuthology.orchestra.run.trial112.stdout:(70/99): python3-grpcio-tools-1.46.7-10.el9.x86 3.0 MB/s | 144 kB 00:00 2026-02-24T18:20:12.645 INFO:teuthology.orchestra.run.trial112.stdout:(71/99): python3-influxdb-5.3.1-1.el9.noarch.rp 2.8 MB/s | 139 kB 00:00 2026-02-24T18:20:12.655 INFO:teuthology.orchestra.run.trial112.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.1 MB/s | 56 kB 00:00 2026-02-24T18:20:12.683 INFO:teuthology.orchestra.run.trial112.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 281 kB/s | 11 kB 00:00 2026-02-24T18:20:12.693 INFO:teuthology.orchestra.run.trial112.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 462 kB/s | 18 kB 00:00 2026-02-24T18:20:12.721 INFO:teuthology.orchestra.run.trial112.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 607 kB/s | 23 kB 00:00 2026-02-24T18:20:12.736 INFO:teuthology.orchestra.run.trial112.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 466 kB/s | 20 kB 00:00 2026-02-24T18:20:12.737 INFO:teuthology.orchestra.run.trial032.stdout:(55/99): protobuf-compiler-3.14.0-17.el9.x86_64 5.8 MB/s | 862 kB 00:00 2026-02-24T18:20:12.760 INFO:teuthology.orchestra.run.trial112.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 509 kB/s | 19 kB 00:00 2026-02-24T18:20:12.766 INFO:teuthology.orchestra.run.trial110.stdout:(13/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 3.2 MB/s | 769 kB 00:00 2026-02-24T18:20:12.781 INFO:teuthology.orchestra.run.trial032.stdout:(56/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 34 MB/s | 19 MB 00:00 2026-02-24T18:20:12.783 INFO:teuthology.orchestra.run.trial112.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 561 kB/s | 26 kB 00:00 2026-02-24T18:20:12.804 INFO:teuthology.orchestra.run.trial032.stdout:(57/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 3.1 MB/s | 17 MB 00:05 2026-02-24T18:20:12.827 INFO:teuthology.orchestra.run.trial112.stdout:(79/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.4 MB/s | 2.0 MB 00:00 2026-02-24T18:20:12.837 INFO:teuthology.orchestra.run.trial032.stdout:(58/99): grpc-data-1.46.7-10.el9.noarch.rpm 351 kB/s | 19 kB 00:00 2026-02-24T18:20:12.844 INFO:teuthology.orchestra.run.trial032.stdout:(59/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 5.1 MB/s | 551 kB 00:00 2026-02-24T18:20:12.846 INFO:teuthology.orchestra.run.trial112.stdout:(80/99): python3-kubernetes-26.1.0-3.el9.noarch 12 MB/s | 1.0 MB 00:00 2026-02-24T18:20:12.855 INFO:teuthology.orchestra.run.trial032.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 2.6 MB/s | 29 kB 00:00 2026-02-24T18:20:12.856 INFO:teuthology.orchestra.run.trial032.stdout:(61/99): luarocks-3.9.2-5.el9.noarch.rpm 2.8 MB/s | 151 kB 00:00 2026-02-24T18:20:12.866 INFO:teuthology.orchestra.run.trial112.stdout:(81/99): python3-logutils-0.3.5-21.el9.noarch.r 556 kB/s | 46 kB 00:00 2026-02-24T18:20:12.867 INFO:teuthology.orchestra.run.trial032.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 5.1 MB/s | 60 kB 00:00 2026-02-24T18:20:12.867 INFO:teuthology.orchestra.run.trial032.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 3.7 MB/s | 43 kB 00:00 2026-02-24T18:20:12.874 INFO:teuthology.orchestra.run.trial112.stdout:(82/99): python3-more-itertools-8.12.0-2.el9.no 1.6 MB/s | 79 kB 00:00 2026-02-24T18:20:12.877 INFO:teuthology.orchestra.run.trial032.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 2.9 MB/s | 32 kB 00:00 2026-02-24T18:20:12.878 INFO:teuthology.orchestra.run.trial032.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 1.3 MB/s | 14 kB 00:00 2026-02-24T18:20:12.879 INFO:teuthology.orchestra.run.trial110.stdout:(14/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.0 MB/s | 351 kB 00:00 2026-02-24T18:20:12.884 INFO:teuthology.orchestra.run.trial032.stdout:(66/99): python3-asyncssh-2.13.2-5.el9.noarch.r 11 MB/s | 548 kB 00:00 2026-02-24T18:20:12.890 INFO:teuthology.orchestra.run.trial032.stdout:(67/99): python3-cheroot-10.0.1-4.el9.noarch.rp 14 MB/s | 173 kB 00:00 2026-02-24T18:20:12.896 INFO:teuthology.orchestra.run.trial032.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 20 MB/s | 254 kB 00:00 2026-02-24T18:20:12.903 INFO:teuthology.orchestra.run.trial032.stdout:(69/99): python3-cherrypy-18.6.1-2.el9.noarch.r 14 MB/s | 358 kB 00:00 2026-02-24T18:20:12.916 INFO:teuthology.orchestra.run.trial032.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 11 MB/s | 139 kB 00:00 2026-02-24T18:20:12.918 INFO:teuthology.orchestra.run.trial110.stdout:(15/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 7.2 MB/s | 17 MB 00:02 2026-02-24T18:20:12.923 INFO:teuthology.orchestra.run.trial112.stdout:(83/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2026-02-24T18:20:12.927 INFO:teuthology.orchestra.run.trial032.stdout:(71/99): python3-grpcio-tools-1.46.7-10.el9.x86 4.6 MB/s | 144 kB 00:00 2026-02-24T18:20:12.927 INFO:teuthology.orchestra.run.trial032.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 4.8 MB/s | 56 kB 00:00 2026-02-24T18:20:12.930 INFO:teuthology.orchestra.run.trial110.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 792 kB/s | 40 kB 00:00 2026-02-24T18:20:12.931 INFO:teuthology.orchestra.run.trial032.stdout:(73/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 49 MB/s | 2.0 MB 00:00 2026-02-24T18:20:12.932 INFO:teuthology.orchestra.run.trial112.stdout:(84/99): python3-natsort-7.1.1-5.el9.noarch.rpm 876 kB/s | 58 kB 00:00 2026-02-24T18:20:12.937 INFO:teuthology.orchestra.run.trial032.stdout:(74/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.1 MB/s | 11 kB 00:00 2026-02-24T18:20:12.938 INFO:teuthology.orchestra.run.trial032.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 1.5 MB/s | 18 kB 00:00 2026-02-24T18:20:12.942 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-24T18:20:12.943 INFO:teuthology.orchestra.run.trial032.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 2.0 MB/s | 23 kB 00:00 2026-02-24T18:20:12.947 INFO:teuthology.orchestra.run.trial032.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 1.9 MB/s | 20 kB 00:00 2026-02-24T18:20:12.949 INFO:teuthology.orchestra.run.trial032.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 1.8 MB/s | 19 kB 00:00 2026-02-24T18:20:12.952 INFO:teuthology.orchestra.run.trial112.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 3.4 MB/s | 272 kB 00:00 2026-02-24T18:20:12.954 INFO:teuthology.orchestra.run.trial032.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 2.4 MB/s | 26 kB 00:00 2026-02-24T18:20:12.959 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-24T18:20:12.963 INFO:teuthology.orchestra.run.trial032.stdout:(80/99): python3-logutils-0.3.5-21.el9.noarch.r 3.2 MB/s | 46 kB 00:00 2026-02-24T18:20:12.966 INFO:teuthology.orchestra.run.trial032.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 6.2 MB/s | 79 kB 00:00 2026-02-24T18:20:12.969 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-24T18:20:12.972 INFO:teuthology.orchestra.run.trial110.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 1.3 MB/s | 72 kB 00:00 2026-02-24T18:20:12.976 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-24T18:20:12.978 INFO:teuthology.orchestra.run.trial032.stdout:(82/99): python3-kubernetes-26.1.0-3.el9.noarch 33 MB/s | 1.0 MB 00:00 2026-02-24T18:20:12.978 INFO:teuthology.orchestra.run.trial032.stdout:(83/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 5.7 MB/s | 86 kB 00:00 2026-02-24T18:20:12.978 INFO:teuthology.orchestra.run.trial032.stdout:(84/99): python3-natsort-7.1.1-5.el9.noarch.rpm 4.6 MB/s | 58 kB 00:00 2026-02-24T18:20:12.989 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-24T18:20:12.990 INFO:teuthology.orchestra.run.trial032.stdout:(85/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 7.5 MB/s | 90 kB 00:00 2026-02-24T18:20:12.991 INFO:teuthology.orchestra.run.trial032.stdout:(86/99): python3-pecan-1.4.2-3.el9.noarch.rpm 20 MB/s | 272 kB 00:00 2026-02-24T18:20:12.992 INFO:teuthology.orchestra.run.trial032.stdout:(87/99): python3-portend-3.1.0-2.el9.noarch.rpm 1.2 MB/s | 16 kB 00:00 2026-02-24T18:20:12.993 INFO:teuthology.orchestra.run.trial112.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 235 kB/s | 16 kB 00:00 2026-02-24T18:20:13.003 INFO:teuthology.orchestra.run.trial112.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2026-02-24T18:20:13.003 INFO:teuthology.orchestra.run.trial032.stdout:(88/99): python3-rsa-4.9-2.el9.noarch.rpm 5.2 MB/s | 59 kB 00:00 2026-02-24T18:20:13.004 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-24T18:20:13.015 INFO:teuthology.orchestra.run.trial110.stdout:(18/99): libquadmath-11.5.0-14.el9.x86_64.rpm 4.2 MB/s | 184 kB 00:00 2026-02-24T18:20:13.016 INFO:teuthology.orchestra.run.trial032.stdout:(89/99): python3-saml-1.16.0-1.el9.noarch.rpm 10 MB/s | 125 kB 00:00 2026-02-24T18:20:13.017 INFO:teuthology.orchestra.run.trial032.stdout:(90/99): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2026-02-24T18:20:13.017 INFO:teuthology.orchestra.run.trial032.stdout:(91/99): python3-repoze-lru-0.7-16.el9.noarch.r 1.1 MB/s | 31 kB 00:00 2026-02-24T18:20:13.017 INFO:teuthology.orchestra.run.trial110.stdout:(19/99): libgfortran-11.5.0-14.el9.x86_64.rpm 8.9 MB/s | 794 kB 00:00 2026-02-24T18:20:13.026 INFO:teuthology.orchestra.run.trial032.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 3.2 MB/s | 36 kB 00:00 2026-02-24T18:20:13.029 INFO:teuthology.orchestra.run.trial032.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 6.9 MB/s | 86 kB 00:00 2026-02-24T18:20:13.030 INFO:teuthology.orchestra.run.trial032.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 18 MB/s | 230 kB 00:00 2026-02-24T18:20:13.038 INFO:teuthology.orchestra.run.trial032.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 7.5 MB/s | 90 kB 00:00 2026-02-24T18:20:13.039 INFO:teuthology.orchestra.run.trial112.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 357 kB/s | 31 kB 00:00 2026-02-24T18:20:13.045 INFO:teuthology.orchestra.run.trial032.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 3.0 MB/s | 48 kB 00:00 2026-02-24T18:20:13.049 INFO:teuthology.orchestra.run.trial032.stdout:(97/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 21 MB/s | 427 kB 00:00 2026-02-24T18:20:13.049 INFO:teuthology.orchestra.run.trial032.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 2.0 MB/s | 22 kB 00:00 2026-02-24T18:20:13.055 INFO:teuthology.orchestra.run.trial112.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 3.0 MB/s | 188 kB 00:00 2026-02-24T18:20:13.061 INFO:teuthology.orchestra.run.trial110.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.4 MB/s | 253 kB 00:00 2026-02-24T18:20:13.062 INFO:teuthology.orchestra.run.trial032.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2026-02-24T18:20:13.063 INFO:teuthology.orchestra.run.trial110.stdout:(21/99): python3-cryptography-36.0.1-5.el9.x86_ 28 MB/s | 1.2 MB 00:00 2026-02-24T18:20:13.064 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:13.064 INFO:teuthology.orchestra.run.trial032.stdout:Total 14 MB/s | 95 MB 00:07 2026-02-24T18:20:13.079 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-24T18:20:13.080 INFO:teuthology.orchestra.run.trial112.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 771 kB/s | 59 kB 00:00 2026-02-24T18:20:13.088 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-24T18:20:13.095 INFO:teuthology.orchestra.run.trial110.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2026-02-24T18:20:13.095 INFO:teuthology.orchestra.run.trial112.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 2.2 MB/s | 125 kB 00:00 2026-02-24T18:20:13.099 INFO:teuthology.orchestra.run.trial110.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 135 kB 00:00 2026-02-24T18:20:13.111 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-24T18:20:13.125 INFO:teuthology.orchestra.run.trial110.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 4.9 MB/s | 150 kB 00:00 2026-02-24T18:20:13.134 INFO:teuthology.orchestra.run.trial110.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 3.6 MB/s | 126 kB 00:00 2026-02-24T18:20:13.135 INFO:teuthology.orchestra.run.trial112.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 446 kB/s | 36 kB 00:00 2026-02-24T18:20:13.139 INFO:teuthology.orchestra.run.trial112.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 1.4 MB/s | 86 kB 00:00 2026-02-24T18:20:13.140 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-24T18:20:13.144 INFO:teuthology.orchestra.run.trial112.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 4.6 MB/s | 230 kB 00:00 2026-02-24T18:20:13.168 INFO:teuthology.orchestra.run.trial110.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 5.0 MB/s | 218 kB 00:00 2026-02-24T18:20:13.170 INFO:teuthology.orchestra.run.trial110.stdout:(27/99): unzip-6.0-59.el9.x86_64.rpm 4.9 MB/s | 182 kB 00:00 2026-02-24T18:20:13.174 INFO:teuthology.orchestra.run.trial112.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 2.3 MB/s | 90 kB 00:00 2026-02-24T18:20:13.190 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-24T18:20:13.196 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-24T18:20:13.199 INFO:teuthology.orchestra.run.trial112.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 890 kB/s | 48 kB 00:00 2026-02-24T18:20:13.200 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-24T18:20:13.201 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-24T18:20:13.207 INFO:teuthology.orchestra.run.trial110.stdout:(28/99): zip-3.0-35.el9.x86_64.rpm 6.7 MB/s | 266 kB 00:00 2026-02-24T18:20:13.210 INFO:teuthology.orchestra.run.trial112.stdout:(97/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.9 MB/s | 427 kB 00:00 2026-02-24T18:20:13.217 INFO:teuthology.orchestra.run.trial112.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 517 kB/s | 22 kB 00:00 2026-02-24T18:20:13.219 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:13.219 INFO:teuthology.orchestra.run.trial018.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T18:20:13.219 INFO:teuthology.orchestra.run.trial018.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T18:20:13.219 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:13.226 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:13.251 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:13.252 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T18:20:13.252 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:13.254 INFO:teuthology.orchestra.run.trial112.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 363 kB/s | 20 kB 00:00 2026-02-24T18:20:13.260 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:13.260 INFO:teuthology.orchestra.run.trial112.stdout:Total 16 MB/s | 95 MB 00:05 2026-02-24T18:20:13.286 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-24T18:20:13.349 INFO:teuthology.orchestra.run.trial018.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-24T18:20:13.352 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:13.366 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:13.367 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:13.367 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T18:20:13.367 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:13.429 INFO:teuthology.orchestra.run.trial110.stdout:(29/99): flexiblas-3.0.4-9.el9.x86_64.rpm 115 kB/s | 30 kB 00:00 2026-02-24T18:20:13.484 INFO:teuthology.orchestra.run.trial110.stdout:(30/99): ceph-mgr-diskprediction-local-19.2.3-6 4.5 MB/s | 7.4 MB 00:01 2026-02-24T18:20:13.492 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:13.529 INFO:teuthology.orchestra.run.trial110.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 147 kB/s | 15 kB 00:00 2026-02-24T18:20:13.530 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:13.530 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:13.697 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:13.735 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:13.736 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:13.819 INFO:teuthology.orchestra.run.trial110.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 650 kB/s | 188 kB 00:00 2026-02-24T18:20:13.907 INFO:teuthology.orchestra.run.trial110.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 479 kB/s | 42 kB 00:00 2026-02-24T18:20:13.948 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:13.948 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:13.981 INFO:teuthology.orchestra.run.trial110.stdout:(34/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 3.9 MB/s | 3.0 MB 00:00 2026-02-24T18:20:13.982 INFO:teuthology.orchestra.run.trial110.stdout:(35/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 495 kB/s | 246 kB 00:00 2026-02-24T18:20:14.065 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:14.077 INFO:teuthology.orchestra.run.trial110.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2026-02-24T18:20:14.090 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:14.091 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:14.091 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T18:20:14.091 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:14.091 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:14.091 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:14.153 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:14.153 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:14.153 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:14.155 INFO:teuthology.orchestra.run.trial018.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:14.159 INFO:teuthology.orchestra.run.trial110.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2026-02-24T18:20:14.160 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-24T18:20:14.179 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-24T18:20:14.180 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:14.242 INFO:teuthology.orchestra.run.trial110.stdout:(38/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 3.0 MB/s | 249 kB 00:00 2026-02-24T18:20:14.322 INFO:teuthology.orchestra.run.trial110.stdout:(39/99): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-24T18:20:14.369 INFO:teuthology.orchestra.run.trial110.stdout:(40/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 11 MB/s | 5.3 MB 00:00 2026-02-24T18:20:14.405 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:14.408 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-24T18:20:14.423 INFO:teuthology.orchestra.run.trial110.stdout:(41/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2026-02-24T18:20:14.438 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-24T18:20:14.444 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-24T18:20:14.456 INFO:teuthology.orchestra.run.trial110.stdout:(42/99): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2026-02-24T18:20:14.457 INFO:teuthology.orchestra.run.trial110.stdout:(43/99): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2026-02-24T18:20:14.465 INFO:teuthology.orchestra.run.trial032.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-24T18:20:14.471 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 5/99 2026-02-24T18:20:14.502 INFO:teuthology.orchestra.run.trial110.stdout:(44/99): python3-markupsafe-1.1.1-12.el9.x86_64 440 kB/s | 35 kB 00:00 2026-02-24T18:20:14.504 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-24T18:20:14.514 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-24T18:20:14.519 INFO:teuthology.orchestra.run.trial032.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-24T18:20:14.521 INFO:teuthology.orchestra.run.trial032.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-24T18:20:14.526 INFO:teuthology.orchestra.run.trial032.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-24T18:20:14.546 INFO:teuthology.orchestra.run.trial110.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 4.9 MB/s | 442 kB 00:00 2026-02-24T18:20:14.549 INFO:teuthology.orchestra.run.trial032.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-24T18:20:14.561 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-24T18:20:14.564 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-24T18:20:14.570 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-24T18:20:14.572 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-24T18:20:14.581 INFO:teuthology.orchestra.run.trial110.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 970 kB/s | 77 kB 00:00 2026-02-24T18:20:14.583 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-24T18:20:14.589 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-24T18:20:14.605 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:14.607 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-24T18:20:14.613 INFO:teuthology.orchestra.run.trial032.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-24T18:20:14.616 INFO:teuthology.orchestra.run.trial032.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-24T18:20:14.624 INFO:teuthology.orchestra.run.trial032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-24T18:20:14.629 INFO:teuthology.orchestra.run.trial110.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-24T18:20:14.638 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-24T18:20:14.643 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-24T18:20:14.648 INFO:teuthology.orchestra.run.trial032.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-24T18:20:14.649 INFO:teuthology.orchestra.run.trial110.stdout:(48/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 32 MB/s | 6.1 MB 00:00 2026-02-24T18:20:14.661 INFO:teuthology.orchestra.run.trial110.stdout:(49/99): python3-pyasn1-0.4.8-7.el9.noarch.rpm 1.9 MB/s | 157 kB 00:00 2026-02-24T18:20:14.664 INFO:teuthology.orchestra.run.trial112.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-24T18:20:14.669 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:14.670 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 5/99 2026-02-24T18:20:14.672 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:14.700 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-24T18:20:14.702 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-24T18:20:14.712 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-24T18:20:14.712 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 23/99 2026-02-24T18:20:14.714 INFO:teuthology.orchestra.run.trial110.stdout:(50/99): python3-pyasn1-modules-0.4.8-7.el9.noa 3.2 MB/s | 277 kB 00:00 2026-02-24T18:20:14.715 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-24T18:20:14.717 INFO:teuthology.orchestra.run.trial112.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-24T18:20:14.720 INFO:teuthology.orchestra.run.trial032.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-24T18:20:14.720 INFO:teuthology.orchestra.run.trial112.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-24T18:20:14.724 INFO:teuthology.orchestra.run.trial112.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-24T18:20:14.725 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-24T18:20:14.729 INFO:teuthology.orchestra.run.trial110.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 678 kB/s | 54 kB 00:00 2026-02-24T18:20:14.730 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-24T18:20:14.733 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-24T18:20:14.747 INFO:teuthology.orchestra.run.trial112.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-24T18:20:14.747 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-24T18:20:14.759 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-24T18:20:14.762 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-24T18:20:14.768 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-24T18:20:14.770 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-24T18:20:14.771 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-24T18:20:14.773 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-24T18:20:14.778 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-24T18:20:14.783 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-24T18:20:14.788 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-24T18:20:14.789 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-24T18:20:14.794 INFO:teuthology.orchestra.run.trial110.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 521 kB/s | 42 kB 00:00 2026-02-24T18:20:14.798 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-24T18:20:14.804 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-24T18:20:14.808 INFO:teuthology.orchestra.run.trial110.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-24T18:20:14.812 INFO:teuthology.orchestra.run.trial112.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-24T18:20:14.815 INFO:teuthology.orchestra.run.trial112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-24T18:20:14.822 INFO:teuthology.orchestra.run.trial112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-24T18:20:14.824 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-24T18:20:14.833 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-24T18:20:14.838 INFO:teuthology.orchestra.run.trial110.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 752 kB/s | 22 kB 00:00 2026-02-24T18:20:14.846 INFO:teuthology.orchestra.run.trial112.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-24T18:20:14.874 INFO:teuthology.orchestra.run.trial110.stdout:(55/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-24T18:20:14.882 INFO:teuthology.orchestra.run.trial110.stdout:(56/99): protobuf-compiler-3.14.0-17.el9.x86_64 19 MB/s | 862 kB 00:00 2026-02-24T18:20:14.887 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-24T18:20:14.896 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-24T18:20:14.900 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-24T18:20:14.909 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 23/99 2026-02-24T18:20:14.911 INFO:teuthology.orchestra.run.trial110.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 673 kB/s | 19 kB 00:00 2026-02-24T18:20:14.912 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-24T18:20:14.915 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-24T18:20:14.916 INFO:teuthology.orchestra.run.trial112.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-24T18:20:14.920 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-24T18:20:14.921 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-24T18:20:14.926 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-24T18:20:14.927 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-24T18:20:14.929 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-24T18:20:14.940 INFO:teuthology.orchestra.run.trial110.stdout:(58/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 8.1 MB/s | 551 kB 00:00 2026-02-24T18:20:14.943 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-24T18:20:14.946 INFO:teuthology.orchestra.run.trial110.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2026-02-24T18:20:14.955 INFO:teuthology.orchestra.run.trial110.stdout:(60/99): python3-asyncssh-2.13.2-5.el9.noarch.r 35 MB/s | 548 kB 00:00 2026-02-24T18:20:14.957 INFO:teuthology.orchestra.run.trial110.stdout:(61/99): python3-autocommand-2.2.2-8.el9.noarch 2.6 MB/s | 29 kB 00:00 2026-02-24T18:20:14.964 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-24T18:20:14.966 INFO:teuthology.orchestra.run.trial110.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 5.5 MB/s | 60 kB 00:00 2026-02-24T18:20:14.967 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-24T18:20:14.968 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-24T18:20:14.969 INFO:teuthology.orchestra.run.trial110.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 3.8 MB/s | 43 kB 00:00 2026-02-24T18:20:14.973 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-24T18:20:14.976 INFO:teuthology.orchestra.run.trial110.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 3.1 MB/s | 32 kB 00:00 2026-02-24T18:20:14.979 INFO:teuthology.orchestra.run.trial110.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 1.3 MB/s | 14 kB 00:00 2026-02-24T18:20:14.984 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-24T18:20:14.988 INFO:teuthology.orchestra.run.trial110.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 15 MB/s | 173 kB 00:00 2026-02-24T18:20:14.993 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-24T18:20:14.999 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-24T18:20:15.003 INFO:teuthology.orchestra.run.trial110.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 17 MB/s | 254 kB 00:00 2026-02-24T18:20:15.003 INFO:teuthology.orchestra.run.trial110.stdout:(68/99): python3-cherrypy-18.6.1-2.el9.noarch.r 15 MB/s | 358 kB 00:00 2026-02-24T18:20:15.017 INFO:teuthology.orchestra.run.trial110.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 11 MB/s | 144 kB 00:00 2026-02-24T18:20:15.018 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-24T18:20:15.027 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-24T18:20:15.031 INFO:teuthology.orchestra.run.trial110.stdout:(70/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 71 MB/s | 2.0 MB 00:00 2026-02-24T18:20:15.032 INFO:teuthology.orchestra.run.trial110.stdout:(71/99): python3-influxdb-5.3.1-1.el9.noarch.rp 8.9 MB/s | 139 kB 00:00 2026-02-24T18:20:15.042 INFO:teuthology.orchestra.run.trial110.stdout:(72/99): python3-isodate-0.6.1-3.el9.noarch.rpm 5.1 MB/s | 56 kB 00:00 2026-02-24T18:20:15.043 INFO:teuthology.orchestra.run.trial110.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.0 MB/s | 11 kB 00:00 2026-02-24T18:20:15.052 INFO:teuthology.orchestra.run.trial110.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 1.7 MB/s | 18 kB 00:00 2026-02-24T18:20:15.054 INFO:teuthology.orchestra.run.trial110.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 2.1 MB/s | 23 kB 00:00 2026-02-24T18:20:15.063 INFO:teuthology.orchestra.run.trial110.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 1.9 MB/s | 20 kB 00:00 2026-02-24T18:20:15.065 INFO:teuthology.orchestra.run.trial110.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 1.7 MB/s | 19 kB 00:00 2026-02-24T18:20:15.073 INFO:teuthology.orchestra.run.trial110.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 2.5 MB/s | 26 kB 00:00 2026-02-24T18:20:15.080 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-24T18:20:15.083 INFO:teuthology.orchestra.run.trial110.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 4.4 MB/s | 46 kB 00:00 2026-02-24T18:20:15.089 INFO:teuthology.orchestra.run.trial110.stdout:(80/99): python3-kubernetes-26.1.0-3.el9.noarch 43 MB/s | 1.0 MB 00:00 2026-02-24T18:20:15.092 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-24T18:20:15.094 INFO:teuthology.orchestra.run.trial110.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 7.0 MB/s | 79 kB 00:00 2026-02-24T18:20:15.101 INFO:teuthology.orchestra.run.trial110.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 7.0 MB/s | 86 kB 00:00 2026-02-24T18:20:15.105 INFO:teuthology.orchestra.run.trial110.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 5.3 MB/s | 58 kB 00:00 2026-02-24T18:20:15.107 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-24T18:20:15.112 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-24T18:20:15.113 INFO:teuthology.orchestra.run.trial110.stdout:(84/99): python3-pecan-1.4.2-3.el9.noarch.rpm 21 MB/s | 272 kB 00:00 2026-02-24T18:20:15.115 INFO:teuthology.orchestra.run.trial110.stdout:(85/99): python3-portend-3.1.0-2.el9.noarch.rpm 1.6 MB/s | 16 kB 00:00 2026-02-24T18:20:15.119 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-24T18:20:15.125 INFO:teuthology.orchestra.run.trial110.stdout:(86/99): python3-repoze-lru-0.7-16.el9.noarch.r 3.0 MB/s | 31 kB 00:00 2026-02-24T18:20:15.126 INFO:teuthology.orchestra.run.trial110.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 7.2 MB/s | 90 kB 00:00 2026-02-24T18:20:15.127 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:15.128 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:15.137 INFO:teuthology.orchestra.run.trial110.stdout:(88/99): python3-routes-2.5.1-5.el9.noarch.rpm 16 MB/s | 188 kB 00:00 2026-02-24T18:20:15.138 INFO:teuthology.orchestra.run.trial110.stdout:(89/99): python3-rsa-4.9-2.el9.noarch.rpm 4.8 MB/s | 59 kB 00:00 2026-02-24T18:20:15.141 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:15.141 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:15.149 INFO:teuthology.orchestra.run.trial110.stdout:(90/99): python3-saml-1.16.0-1.el9.noarch.rpm 10 MB/s | 125 kB 00:00 2026-02-24T18:20:15.149 INFO:teuthology.orchestra.run.trial110.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 3.0 MB/s | 36 kB 00:00 2026-02-24T18:20:15.159 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-24T18:20:15.159 INFO:teuthology.orchestra.run.trial110.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 8.2 MB/s | 86 kB 00:00 2026-02-24T18:20:15.162 INFO:teuthology.orchestra.run.trial110.stdout:(93/99): python3-webob-1.8.8-2.el9.noarch.rpm 18 MB/s | 230 kB 00:00 2026-02-24T18:20:15.171 INFO:teuthology.orchestra.run.trial110.stdout:(94/99): python3-websocket-client-1.2.3-2.el9.n 7.7 MB/s | 90 kB 00:00 2026-02-24T18:20:15.176 INFO:teuthology.orchestra.run.trial110.stdout:(95/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 30 MB/s | 427 kB 00:00 2026-02-24T18:20:15.181 INFO:teuthology.orchestra.run.trial110.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 4.5 MB/s | 48 kB 00:00 2026-02-24T18:20:15.187 INFO:teuthology.orchestra.run.trial110.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 1.9 MB/s | 22 kB 00:00 2026-02-24T18:20:15.192 INFO:teuthology.orchestra.run.trial110.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 1.9 MB/s | 20 kB 00:00 2026-02-24T18:20:15.196 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:15.245 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-24T18:20:15.246 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:15.268 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:15.279 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:15.281 INFO:teuthology.orchestra.run.trial110.stdout:(99/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-24T18:20:15.283 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:15.283 INFO:teuthology.orchestra.run.trial110.stdout:Total 17 MB/s | 95 MB 00:05 2026-02-24T18:20:15.295 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-24T18:20:15.305 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-24T18:20:15.309 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-24T18:20:15.314 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-24T18:20:15.316 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-24T18:20:15.344 INFO:teuthology.orchestra.run.trial032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-24T18:20:15.387 INFO:teuthology.orchestra.run.trial032.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-24T18:20:15.394 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-24T18:20:15.398 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-24T18:20:15.403 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-24T18:20:15.407 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-24T18:20:15.414 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-24T18:20:15.418 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-24T18:20:15.446 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-24T18:20:15.457 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-24T18:20:15.478 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-24T18:20:15.488 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-24T18:20:15.491 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-24T18:20:15.492 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-24T18:20:15.495 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:15.496 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:15.497 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-24T18:20:15.499 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-24T18:20:15.518 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:15.518 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:15.518 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T18:20:15.519 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:15.519 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:15.519 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:15.526 INFO:teuthology.orchestra.run.trial112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-24T18:20:15.570 INFO:teuthology.orchestra.run.trial112.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-24T18:20:15.576 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-24T18:20:15.580 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-24T18:20:15.585 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-24T18:20:15.589 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-24T18:20:15.596 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-24T18:20:15.599 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-24T18:20:15.619 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:15.627 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-24T18:20:15.636 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:15.642 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:15.662 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:15.670 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-24T18:20:15.716 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:15.716 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-24T18:20:15.740 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-24T18:20:15.744 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-24T18:20:15.756 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:15.756 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:15.802 INFO:teuthology.orchestra.run.trial032.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-24T18:20:15.804 INFO:teuthology.orchestra.run.trial032.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-24T18:20:15.825 INFO:teuthology.orchestra.run.trial032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-24T18:20:15.890 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-24T18:20:15.913 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-24T18:20:15.917 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-24T18:20:15.975 INFO:teuthology.orchestra.run.trial112.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-24T18:20:15.977 INFO:teuthology.orchestra.run.trial112.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-24T18:20:15.998 INFO:teuthology.orchestra.run.trial112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-24T18:20:16.151 INFO:teuthology.orchestra.run.trial032.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-24T18:20:16.174 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:16.175 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:16.224 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-24T18:20:16.318 INFO:teuthology.orchestra.run.trial112.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-24T18:20:16.362 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:16.362 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-24T18:20:16.363 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-24T18:20:16.364 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-24T18:20:16.365 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-24T18:20:16.366 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-24T18:20:16.367 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-24T18:20:16.368 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-24T18:20:16.369 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-24T18:20:16.370 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-24T18:20:16.389 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:16.430 INFO:teuthology.orchestra.run.trial018.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:16.431 INFO:teuthology.orchestra.run.trial018.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T18:20:16.432 INFO:teuthology.orchestra.run.trial018.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T18:20:16.433 INFO:teuthology.orchestra.run.trial018.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T18:20:16.434 INFO:teuthology.orchestra.run.trial018.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-24T18:20:16.435 INFO:teuthology.orchestra.run.trial018.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T18:20:16.436 INFO:teuthology.orchestra.run.trial018.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T18:20:16.437 INFO:teuthology.orchestra.run.trial018.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T18:20:16.438 INFO:teuthology.orchestra.run.trial018.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T18:20:16.439 INFO:teuthology.orchestra.run.trial018.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T18:20:16.439 INFO:teuthology.orchestra.run.trial018.stdout: zip-3.0-35.el9.x86_64 2026-02-24T18:20:16.439 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:16.439 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:16.489 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-base 2026-02-24T18:20:16.630 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:16.632 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-24T18:20:16.663 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-24T18:20:16.668 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-24T18:20:16.689 INFO:teuthology.orchestra.run.trial110.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-24T18:20:16.695 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 5/99 2026-02-24T18:20:16.729 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-24T18:20:16.738 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-24T18:20:16.743 INFO:teuthology.orchestra.run.trial110.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-24T18:20:16.746 INFO:teuthology.orchestra.run.trial110.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-24T18:20:16.750 INFO:teuthology.orchestra.run.trial110.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-24T18:20:16.773 INFO:teuthology.orchestra.run.trial110.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-24T18:20:16.786 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-24T18:20:16.789 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-24T18:20:16.795 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-24T18:20:16.798 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-24T18:20:16.809 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-24T18:20:16.810 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:41 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:16.815 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-24T18:20:16.840 INFO:teuthology.orchestra.run.trial110.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-24T18:20:16.843 INFO:teuthology.orchestra.run.trial110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-24T18:20:16.850 INFO:teuthology.orchestra.run.trial110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-24T18:20:16.874 INFO:teuthology.orchestra.run.trial110.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-24T18:20:16.876 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:16.894 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:16.895 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:16.895 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:16.897 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-24T18:20:16.915 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-24T18:20:16.917 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install cephadm 2026-02-24T18:20:16.925 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-24T18:20:16.925 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-24T18:20:16.932 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-24T18:20:16.937 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 23/99 2026-02-24T18:20:16.941 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-24T18:20:16.945 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-24T18:20:16.946 INFO:teuthology.orchestra.run.trial110.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-24T18:20:16.951 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-24T18:20:16.956 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-24T18:20:16.959 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-24T18:20:16.960 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-24T18:20:16.974 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-24T18:20:16.994 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-24T18:20:16.999 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-24T18:20:17.004 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-24T18:20:17.015 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-24T18:20:17.024 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-24T18:20:17.030 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-24T18:20:17.035 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-24T18:20:17.044 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-24T18:20:17.048 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-24T18:20:17.050 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-24T18:20:17.059 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-24T18:20:17.065 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-24T18:20:17.067 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-24T18:20:17.075 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-24T18:20:17.081 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-24T18:20:17.094 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-24T18:20:17.096 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-24T18:20:17.110 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-24T18:20:17.113 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-24T18:20:17.125 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-24T18:20:17.140 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-24T18:20:17.145 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-24T18:20:17.146 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-24T18:20:17.152 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-24T18:20:17.152 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-24T18:20:17.155 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-24T18:20:17.157 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-24T18:20:17.174 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.175 INFO:teuthology.orchestra.run.trial032.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T18:20:17.175 INFO:teuthology.orchestra.run.trial032.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T18:20:17.175 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:17.182 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.183 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-24T18:20:17.192 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-24T18:20:17.192 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-24T18:20:17.208 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.208 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T18:20:17.208 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:17.215 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-24T18:20:17.234 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:42 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:17.242 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-24T18:20:17.244 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-24T18:20:17.295 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-24T18:20:17.301 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-24T18:20:17.303 INFO:teuthology.orchestra.run.trial018.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:17.304 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-24T18:20:17.305 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-24T18:20:17.306 INFO:teuthology.orchestra.run.trial032.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-24T18:20:17.308 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:17.321 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:17.322 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:17.322 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:17.323 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.323 INFO:teuthology.orchestra.run.trial112.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T18:20:17.323 INFO:teuthology.orchestra.run.trial112.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T18:20:17.323 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:17.324 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:17.324 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:17.324 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T18:20:17.324 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:17.331 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.344 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T18:20:17.355 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:17.355 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T18:20:17.355 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:17.388 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-24T18:20:17.451 INFO:teuthology.orchestra.run.trial112.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-24T18:20:17.453 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:17.467 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:17.467 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:17.467 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T18:20:17.467 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:17.516 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-24T18:20:17.526 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-24T18:20:17.530 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-24T18:20:17.535 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-24T18:20:17.537 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-24T18:20:17.565 INFO:teuthology.orchestra.run.trial110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-24T18:20:17.608 INFO:teuthology.orchestra.run.trial110.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-24T18:20:17.614 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-24T18:20:17.618 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-24T18:20:17.623 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-24T18:20:17.627 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-24T18:20:17.634 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-24T18:20:17.638 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-24T18:20:17.660 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:42 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:17.665 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-24T18:20:17.677 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-24T18:20:17.711 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-24T18:20:17.744 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:17.745 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:17.746 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:17.746 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 145 k 2026-02-24T18:20:17.746 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 431 k 2026-02-24T18:20:17.746 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:17.933 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-24T18:20:17.956 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-24T18:20:17.960 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-24T18:20:18.019 INFO:teuthology.orchestra.run.trial110.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-24T18:20:18.020 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:18.021 INFO:teuthology.orchestra.run.trial110.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-24T18:20:18.042 INFO:teuthology.orchestra.run.trial110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:18.046 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:18.104 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:18.105 INFO:teuthology.orchestra.run.trial032.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:18.110 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-24T18:20:18.130 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-24T18:20:18.131 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:18.167 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:18.193 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:18.193 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:18.193 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T18:20:18.193 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:18.193 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:18.194 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:18.195 INFO:teuthology.orchestra.run.trial018.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 324 kB/s | 145 kB 00:00 2026-02-24T18:20:18.195 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:18.195 INFO:teuthology.orchestra.run.trial018.stdout:Total 323 kB/s | 145 kB 00:00 2026-02-24T18:20:18.195 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:18.198 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:18.198 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:18.230 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:18.230 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:18.252 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:18.254 INFO:teuthology.orchestra.run.trial112.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:18.259 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-24T18:20:18.278 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-24T18:20:18.280 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:18.285 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:18.287 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:18.309 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:18.309 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:18.310 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T18:20:18.310 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:18.364 INFO:teuthology.orchestra.run.trial110.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-24T18:20:18.437 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-24T18:20:18.623 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:18.626 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:18.639 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:18.639 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:18.640 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:18.640 INFO:teuthology.orchestra.run.trial018.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:18.640 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:18.640 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:18.668 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr 2026-02-24T18:20:18.774 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:18.777 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:18.990 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:43 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:19.055 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:19.074 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:19.074 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:19.075 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:19.080 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:19.081 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:19.094 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:19.095 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:19.096 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T18:20:19.102 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-24T18:20:19.119 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-24T18:20:19.129 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-24T18:20:19.136 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-24T18:20:19.149 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-24T18:20:19.151 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:19.164 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-24T18:20:19.201 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-24T18:20:19.202 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:19.223 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:19.224 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.224 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T18:20:19.224 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:19.224 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:19.224 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:19.234 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:19.235 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:19.235 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:19.239 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-24T18:20:19.248 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-24T18:20:19.248 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:19.249 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:19.271 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-24T18:20:19.300 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-24T18:20:19.304 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:19.350 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-24T18:20:19.354 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-24T18:20:19.355 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:19.356 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-24T18:20:19.360 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-24T18:20:19.361 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:19.377 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:19.379 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:19.379 INFO:teuthology.orchestra.run.trial110.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T18:20:19.379 INFO:teuthology.orchestra.run.trial110.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T18:20:19.379 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:19.386 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:19.388 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:19.411 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:44 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:19.412 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:20:19.412 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T18:20:19.412 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:19.445 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-24T18:20:19.456 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:19.458 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:19.477 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:19.480 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:19.495 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:19.496 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:19.496 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:19.509 INFO:teuthology.orchestra.run.trial110.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-24T18:20:19.511 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:19.518 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T18:20:19.525 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:20:19.525 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.525 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T18:20:19.525 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:19.583 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:19.603 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:19.604 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.604 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T18:20:19.604 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:19.604 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:19.604 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:19.605 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:19.607 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:19.622 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:19.629 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:19.730 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:19.752 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:19.753 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:19.753 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T18:20:19.753 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:19.753 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:19.753 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:19.772 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:19.830 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:44 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:19.896 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:19.914 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:19.915 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:19.915 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:19.936 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr-rook 2026-02-24T18:20:20.249 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:20.253 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:20:20.275 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:20.294 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:20.294 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-24T18:20:20.294 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-24T18:20:20.294 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-24T18:20:20.294 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-24T18:20:20.295 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-24T18:20:20.297 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-24T18:20:20.298 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-24T18:20:20.318 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:20.334 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:20.336 INFO:teuthology.orchestra.run.trial110.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:20:20.337 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:20.337 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:20.337 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:20.340 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-24T18:20:20.359 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T18:20:20.361 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-24T18:20:20.361 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:20.370 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-24T18:20:20.370 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.371 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T18:20:20.372 INFO:teuthology.orchestra.run.trial032.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T18:20:20.373 INFO:teuthology.orchestra.run.trial032.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T18:20:20.374 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T18:20:20.375 INFO:teuthology.orchestra.run.trial032.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T18:20:20.376 INFO:teuthology.orchestra.run.trial032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T18:20:20.377 INFO:teuthology.orchestra.run.trial032.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T18:20:20.378 INFO:teuthology.orchestra.run.trial032.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T18:20:20.378 INFO:teuthology.orchestra.run.trial032.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T18:20:20.378 INFO:teuthology.orchestra.run.trial032.stdout: zip-3.0-35.el9.x86_64 2026-02-24T18:20:20.378 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:20.378 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-24T18:20:20.428 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-24T18:20:20.429 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-24T18:20:20.430 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-24T18:20:20.431 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-24T18:20:20.432 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-24T18:20:20.432 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-24T18:20:20.432 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-24T18:20:20.432 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-base 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.500 INFO:teuthology.orchestra.run.trial112.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.501 INFO:teuthology.orchestra.run.trial112.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T18:20:20.502 INFO:teuthology.orchestra.run.trial112.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T18:20:20.503 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T18:20:20.504 INFO:teuthology.orchestra.run.trial112.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T18:20:20.505 INFO:teuthology.orchestra.run.trial112.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: zip-3.0-35.el9.x86_64 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:20.506 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:20.554 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-base 2026-02-24T18:20:20.678 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:20.744 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:20.757 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:40 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:20.762 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:20.763 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:20.764 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:20.785 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-fuse 2026-02-24T18:20:20.823 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:20.842 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:20.842 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:20.842 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:20.859 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:20:20.862 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:20.864 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install cephadm 2026-02-24T18:20:20.872 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:44 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:20.937 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:20.956 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:20.956 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:20.957 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:20.977 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install cephadm 2026-02-24T18:20:21.097 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:46 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:21.181 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:21.182 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 1.2 M 2026-02-24T18:20:21.183 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 3.6 M 2026-02-24T18:20:21.184 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:21.189 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:41 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:21.257 INFO:teuthology.orchestra.run.trial032.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:21.275 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:21.276 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:21.276 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:21.292 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:21.298 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T18:20:21.333 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:20:21.334 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:21.347 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:20:21.348 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:21.357 INFO:teuthology.orchestra.run.trial112.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:21.376 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:21.376 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:21.376 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:21.397 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T18:20:21.402 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:20:21.451 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-24T18:20:21.452 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:20:21.474 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:21.485 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:21.623 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:41 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:21.709 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:21.709 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:21.710 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 145 k 2026-02-24T18:20:21.711 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 431 k 2026-02-24T18:20:21.711 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:21.711 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:20:21.711 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:21.712 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:21.733 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:20:21.733 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:21.733 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T18:20:21.733 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:21.734 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:20:21.734 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:21.796 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:21.797 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 145 k 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 431 k 2026-02-24T18:20:21.798 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:21.811 INFO:teuthology.orchestra.run.trial018.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2026-02-24T18:20:21.812 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:21.812 INFO:teuthology.orchestra.run.trial018.stdout:Total 1.8 MB/s | 1.2 MB 00:00 2026-02-24T18:20:21.812 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:21.816 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:21.816 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:21.842 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:21.852 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:21.853 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:20:21.862 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:21.881 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:21.957 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:21.975 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:22.152 INFO:teuthology.orchestra.run.trial032.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 329 kB/s | 145 kB 00:00 2026-02-24T18:20:22.153 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:22.153 INFO:teuthology.orchestra.run.trial032.stdout:Total 328 kB/s | 145 kB 00:00 2026-02-24T18:20:22.153 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:22.157 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:22.157 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:22.189 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:22.190 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:22.220 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:22.245 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:22.246 INFO:teuthology.orchestra.run.trial112.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 324 kB/s | 145 kB 00:00 2026-02-24T18:20:22.247 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:22.247 INFO:teuthology.orchestra.run.trial112.stdout:Total 323 kB/s | 145 kB 00:00 2026-02-24T18:20:22.247 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:22.247 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.251 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:22.251 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:22.270 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.270 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:22.271 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T18:20:22.271 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:22.282 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:22.282 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:22.282 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:22.283 INFO:teuthology.orchestra.run.trial018.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.283 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:22.283 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:22.283 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:22.284 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:22.310 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-volume 2026-02-24T18:20:22.338 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:22.339 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.361 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.361 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:22.361 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T18:20:22.361 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-24T18:20:22.549 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-24T18:20:22.550 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-24T18:20:22.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-24T18:20:22.553 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-24T18:20:22.554 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-24T18:20:22.554 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-24T18:20:22.554 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:22.562 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:22.590 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.624 INFO:teuthology.orchestra.run.trial110.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T18:20:22.625 INFO:teuthology.orchestra.run.trial110.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: zip-3.0-35.el9.x86_64 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:22.626 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:22.627 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:22.683 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-base 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:22.689 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:22.691 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:22.709 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:22.710 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:22.710 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:22.715 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr 2026-02-24T18:20:22.731 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librados-devel 2026-02-24T18:20:22.910 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:42 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:22.978 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:22.997 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:22.998 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:22.998 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:23.007 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:46 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:23.019 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T18:20:23.036 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:23.047 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:23.073 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:23.092 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:23.093 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:23.093 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:23.103 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:23.114 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install cephadm 2026-02-24T18:20:23.121 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:23.122 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:23.122 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:23.133 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:23.134 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 126 k 2026-02-24T18:20:23.135 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 456 k 2026-02-24T18:20:23.135 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:23.144 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T18:20:23.338 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:43 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:23.404 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:23.423 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:23.424 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:23.424 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:23.433 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:46 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:23.444 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T18:20:23.459 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:23.500 INFO:teuthology.orchestra.run.trial110.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:23.517 INFO:teuthology.orchestra.run.trial018.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 331 kB/s | 126 kB 00:00 2026-02-24T18:20:23.517 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:23.517 INFO:teuthology.orchestra.run.trial018.stdout:Total 330 kB/s | 126 kB 00:00 2026-02-24T18:20:23.517 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:23.519 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:23.520 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:23.520 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:23.521 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:23.521 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:23.523 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:23.541 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T18:20:23.542 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:23.542 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:23.543 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:23.552 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:23.553 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:23.563 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T18:20:23.608 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:23.625 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:23.712 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:23.765 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:43 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:23.776 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:23.776 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:23.776 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:23.776 INFO:teuthology.orchestra.run.trial018.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:23.776 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:23.777 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:23.805 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install libcephfs2 2026-02-24T18:20:23.831 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:23.850 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:23.851 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:23.852 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:23.861 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:46 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:23.872 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr-rook 2026-02-24T18:20:23.876 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:23.941 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:23.946 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:23.946 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:23.946 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:20:23.946 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:23.946 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:23.947 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:23.948 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 145 k 2026-02-24T18:20:23.948 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 431 k 2026-02-24T18:20:23.948 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:23.960 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:23.960 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:23.961 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:23.982 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr-rook 2026-02-24T18:20:24.121 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:24.188 INFO:teuthology.orchestra.run.trial018.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:24.197 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:44 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:24.207 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:24.208 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:24.208 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:24.229 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install libcephfs-devel 2026-02-24T18:20:24.263 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:24.282 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:24.283 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:24.283 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:24.293 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:24.304 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T18:20:24.359 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:24.377 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:24.378 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:24.378 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:24.399 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T18:20:24.414 INFO:teuthology.orchestra.run.trial110.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 311 kB/s | 145 kB 00:00 2026-02-24T18:20:24.414 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:24.415 INFO:teuthology.orchestra.run.trial110.stdout:Total 311 kB/s | 145 kB 00:00 2026-02-24T18:20:24.415 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:24.419 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:24.419 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:24.451 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:24.451 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:24.507 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:24.508 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:24.529 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:24.530 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:24.530 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T18:20:24.530 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:24.542 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:24.622 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:44 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:24.627 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:24.627 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:24.628 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 34 k 2026-02-24T18:20:24.629 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 153 k 2026-02-24T18:20:24.629 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:24.688 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:24.707 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:24.707 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:24.708 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:24.716 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:24.729 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-fuse 2026-02-24T18:20:24.781 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:24.800 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:24.800 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:24.800 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:24.821 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-fuse 2026-02-24T18:20:24.844 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:20:24.845 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:24.845 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:24.845 INFO:teuthology.orchestra.run.trial110.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:24.845 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:24.845 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:24.872 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr 2026-02-24T18:20:24.919 INFO:teuthology.orchestra.run.trial018.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 115 kB/s | 34 kB 00:00 2026-02-24T18:20:24.920 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:24.920 INFO:teuthology.orchestra.run.trial018.stdout:Total 115 kB/s | 34 kB 00:00 2026-02-24T18:20:24.920 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:24.922 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:24.922 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:24.924 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:24.924 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:24.945 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:24.962 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:25.022 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:25.047 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:25.083 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:25.111 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librados2 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:25.131 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:25.132 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 1.2 M 2026-02-24T18:20:25.132 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 3.6 M 2026-02-24T18:20:25.132 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:25.136 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:25.188 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:25.221 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:25.221 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:25.221 INFO:teuthology.orchestra.run.trial112.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:25.221 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 1.2 M 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 3.6 M 2026-02-24T18:20:25.222 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:25.253 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:25.272 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:25.272 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:25.273 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:25.294 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T18:20:25.421 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:25.488 INFO:teuthology.orchestra.run.trial018.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:25.507 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:25.508 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:25.508 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:25.529 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librbd1 2026-02-24T18:20:25.612 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:25.678 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:25.697 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:25.698 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:25.698 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:25.719 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T18:20:25.844 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:25.872 INFO:teuthology.orchestra.run.trial112.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2026-02-24T18:20:25.873 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:25.873 INFO:teuthology.orchestra.run.trial112.stdout:Total 1.8 MB/s | 1.2 MB 00:00 2026-02-24T18:20:25.873 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:25.876 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:25.876 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:25.909 INFO:teuthology.orchestra.run.trial018.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:25.913 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:25.913 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:25.928 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:25.929 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:25.929 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:25.951 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rados 2026-02-24T18:20:26.020 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:26.034 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:26.038 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:26.100 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:26.118 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:26.119 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:26.119 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:26.140 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr-rook 2026-02-24T18:20:26.268 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:26.276 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:26.333 INFO:teuthology.orchestra.run.trial018.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:26.337 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:26.338 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:26.338 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:26.338 INFO:teuthology.orchestra.run.trial112.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:26.338 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:26.338 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:26.352 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:26.353 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:26.353 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:26.366 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-volume 2026-02-24T18:20:26.374 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rgw 2026-02-24T18:20:26.453 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:26.518 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:26.537 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:26.537 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:26.538 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:26.560 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T18:20:26.644 INFO:teuthology.orchestra.run.trial032.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 780 kB/s | 1.2 MB 00:01 2026-02-24T18:20:26.645 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:26.645 INFO:teuthology.orchestra.run.trial032.stdout:Total 780 kB/s | 1.2 MB 00:01 2026-02-24T18:20:26.645 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:26.649 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:26.649 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:26.682 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:26.686 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:26.686 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:26.690 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:26.749 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:26.755 INFO:teuthology.orchestra.run.trial018.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:26.768 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:26.769 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:26.769 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:26.774 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:26.774 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:26.774 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:26.791 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librados-devel 2026-02-24T18:20:26.792 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:26.796 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-cephfs 2026-02-24T18:20:26.811 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:26.882 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:26.948 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:26.966 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:26.967 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:26.967 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:26.988 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-fuse 2026-02-24T18:20:27.030 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:27.093 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:27.094 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:27.094 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:27.094 INFO:teuthology.orchestra.run.trial032.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:27.094 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:27.094 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:27.110 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:27.114 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:27.123 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-volume 2026-02-24T18:20:27.181 INFO:teuthology.orchestra.run.trial018.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:27.196 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:27.197 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:27.198 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:27.198 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 126 k 2026-02-24T18:20:27.198 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 456 k 2026-02-24T18:20:27.198 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:27.200 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:27.201 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:27.201 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:27.225 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rbd 2026-02-24T18:20:27.307 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:27.394 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:27.395 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 1.2 M 2026-02-24T18:20:27.396 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 3.6 M 2026-02-24T18:20:27.397 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:27.440 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:27.505 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:27.524 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:27.524 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:27.524 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:27.541 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:27.545 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librados-devel 2026-02-24T18:20:27.607 INFO:teuthology.orchestra.run.trial018.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:27.625 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:27.626 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:20:27.626 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:27.649 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-fuse 2026-02-24T18:20:27.678 INFO:teuthology.orchestra.run.trial112.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 263 kB/s | 126 kB 00:00 2026-02-24T18:20:27.679 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:27.679 INFO:teuthology.orchestra.run.trial112.stdout:Total 262 kB/s | 126 kB 00:00 2026-02-24T18:20:27.679 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:27.682 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:27.682 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:27.714 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:27.714 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:27.768 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:27.786 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:27.858 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:27.872 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:27.934 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:27.934 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:27.935 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:27.935 INFO:teuthology.orchestra.run.trial112.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:27.935 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:27.935 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:27.942 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:27.942 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:27.943 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 126 k 2026-02-24T18:20:27.944 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 456 k 2026-02-24T18:20:27.944 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:27.963 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install libcephfs2 2026-02-24T18:20:27.964 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:28.049 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:28.050 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 85 k 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 227 k 2026-02-24T18:20:28.051 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:28.086 INFO:teuthology.orchestra.run.trial110.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.7 MB/s | 1.2 MB 00:00 2026-02-24T18:20:28.087 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:28.087 INFO:teuthology.orchestra.run.trial110.stdout:Total 1.7 MB/s | 1.2 MB 00:00 2026-02-24T18:20:28.087 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:28.090 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:28.091 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:28.127 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:28.127 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:28.230 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:28.248 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.285 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:28.351 INFO:teuthology.orchestra.run.trial112.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:28.369 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:28.370 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:28.370 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:28.387 INFO:teuthology.orchestra.run.trial032.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 285 kB/s | 126 kB 00:00 2026-02-24T18:20:28.388 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:28.388 INFO:teuthology.orchestra.run.trial032.stdout:Total 283 kB/s | 126 kB 00:00 2026-02-24T18:20:28.388 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:28.390 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install libcephfs-devel 2026-02-24T18:20:28.391 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:28.391 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:28.423 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:28.423 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:28.428 INFO:teuthology.orchestra.run.trial018.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 225 kB/s | 85 kB 00:00 2026-02-24T18:20:28.429 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:28.429 INFO:teuthology.orchestra.run.trial018.stdout:Total 225 kB/s | 85 kB 00:00 2026-02-24T18:20:28.429 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:28.433 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:28.433 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:28.463 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:28.464 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:28.479 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:28.491 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.496 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.515 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:28.532 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.552 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.552 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:28.552 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:28.552 INFO:teuthology.orchestra.run.trial110.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:28.552 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:28.553 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:28.582 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-volume 2026-02-24T18:20:28.585 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.624 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.647 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.647 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:28.647 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:28.648 INFO:teuthology.orchestra.run.trial032.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:28.648 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:28.648 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:28.675 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install libcephfs2 2026-02-24T18:20:28.687 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:28.687 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:28.687 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:28.688 INFO:teuthology.orchestra.run.trial018.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:28.688 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:28.688 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:28.707 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:28.719 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-mirror 2026-02-24T18:20:28.793 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:28.794 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:28.795 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:28.795 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 34 k 2026-02-24T18:20:28.795 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 153 k 2026-02-24T18:20:28.795 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:28.902 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:28.969 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:20:28.987 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:28.988 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:28.988 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:28.995 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:29.010 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librados-devel 2026-02-24T18:20:29.030 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:29.061 INFO:teuthology.orchestra.run.trial032.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:29.080 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:29.080 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:29.081 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:29.102 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install libcephfs-devel 2026-02-24T18:20:29.105 INFO:teuthology.orchestra.run.trial112.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 108 kB/s | 34 kB 00:00 2026-02-24T18:20:29.105 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:29.105 INFO:teuthology.orchestra.run.trial112.stdout:Total 108 kB/s | 34 kB 00:00 2026-02-24T18:20:29.105 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:29.107 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:29.107 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:29.109 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:29.109 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:29.114 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:29.115 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:29.116 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:29.117 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:29.117 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 3.1 M 2026-02-24T18:20:29.117 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 13 M 2026-02-24T18:20:29.117 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:29.131 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:29.150 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:29.210 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:29.272 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:29.272 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:29.272 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:29.272 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:29.272 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:29.273 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:29.299 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librados2 2026-02-24T18:20:29.329 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:29.413 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:29.414 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:29.415 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:29.415 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:29.415 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 126 k 2026-02-24T18:20:29.415 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 456 k 2026-02-24T18:20:29.415 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:29.420 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:29.504 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:29.504 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:29.504 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:20:29.504 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 34 k 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 153 k 2026-02-24T18:20:29.505 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:29.614 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:53 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:29.679 INFO:teuthology.orchestra.run.trial112.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:29.697 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:29.698 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:29.698 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:29.719 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librbd1 2026-02-24T18:20:29.889 INFO:teuthology.orchestra.run.trial032.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 87 kB/s | 34 kB 00:00 2026-02-24T18:20:29.889 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:29.889 INFO:teuthology.orchestra.run.trial032.stdout:Total 87 kB/s | 34 kB 00:00 2026-02-24T18:20:29.890 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:29.892 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:29.892 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:29.894 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:29.894 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:29.909 INFO:teuthology.orchestra.run.trial018.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 3.9 MB/s | 3.1 MB 00:00 2026-02-24T18:20:29.909 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:29.910 INFO:teuthology.orchestra.run.trial018.stdout:Total 3.9 MB/s | 3.1 MB 00:00 2026-02-24T18:20:29.910 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:29.913 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:29.913 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:29.916 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:29.923 INFO:teuthology.orchestra.run.trial110.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 248 kB/s | 126 kB 00:00 2026-02-24T18:20:29.924 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:29.924 INFO:teuthology.orchestra.run.trial110.stdout:Total 248 kB/s | 126 kB 00:00 2026-02-24T18:20:29.924 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:29.926 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:29.926 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:29.934 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:29.953 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:29.953 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:29.958 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:29.958 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:29.996 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.014 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:30.031 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:30.031 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:30.059 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:30.087 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librados2 2026-02-24T18:20:30.097 INFO:teuthology.orchestra.run.trial112.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.116 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:30.117 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:30.117 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:30.118 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.139 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rados 2026-02-24T18:20:30.148 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:30.149 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.172 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.172 INFO:teuthology.orchestra.run.trial018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:30.172 INFO:teuthology.orchestra.run.trial018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T18:20:30.173 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:30.173 INFO:teuthology.orchestra.run.trial018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:30.173 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:30.179 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:30.208 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install libcephfs2 2026-02-24T18:20:30.399 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:30.454 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:30.463 INFO:teuthology.orchestra.run.trial032.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.482 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:30.483 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:30.483 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:30.490 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:30.490 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:30.491 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:30.491 INFO:teuthology.orchestra.run.trial018.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:30.491 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:30.491 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:30.504 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librbd1 2026-02-24T18:20:30.520 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-nbd 2026-02-24T18:20:30.520 INFO:teuthology.orchestra.run.trial112.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.528 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:53 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:30.538 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:30.539 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:30.539 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:30.561 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rgw 2026-02-24T18:20:30.593 INFO:teuthology.orchestra.run.trial110.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.612 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:30.612 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:30.613 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:30.634 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install libcephfs-devel 2026-02-24T18:20:30.818 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:30.834 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:30.872 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:30.884 INFO:teuthology.orchestra.run.trial032.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.903 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:30.903 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:30.903 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:30.918 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:30.920 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:30.921 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:30.921 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:30.921 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 170 k 2026-02-24T18:20:30.921 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 490 k 2026-02-24T18:20:30.921 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:30.925 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rados 2026-02-24T18:20:30.937 INFO:teuthology.orchestra.run.trial112.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:30.950 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:53 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:30.956 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:30.957 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:30.957 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:30.978 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-cephfs 2026-02-24T18:20:31.034 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:31.035 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:31.036 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:31.036 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:31.036 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 34 k 2026-02-24T18:20:31.036 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 153 k 2026-02-24T18:20:31.036 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:31.243 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:31.293 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:31.308 INFO:teuthology.orchestra.run.trial032.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:31.327 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:31.327 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:31.327 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:31.349 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rgw 2026-02-24T18:20:31.358 INFO:teuthology.orchestra.run.trial112.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:31.364 INFO:teuthology.orchestra.run.trial018.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 384 kB/s | 170 kB 00:00 2026-02-24T18:20:31.364 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:31.365 INFO:teuthology.orchestra.run.trial018.stdout:Total 383 kB/s | 170 kB 00:00 2026-02-24T18:20:31.365 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:31.369 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:31.369 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:31.370 INFO:teuthology.orchestra.run.trial110.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 100 kB/s | 34 kB 00:00 2026-02-24T18:20:31.370 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:31.371 INFO:teuthology.orchestra.run.trial110.stdout:Total 100 kB/s | 34 kB 00:00 2026-02-24T18:20:31.371 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:31.373 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:31.373 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:31.376 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:31.376 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:31.377 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:31.378 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:31.378 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:31.397 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:31.400 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rbd 2026-02-24T18:20:31.401 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:31.401 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:31.415 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.456 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:31.474 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.475 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:31.536 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:31.561 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.563 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librados2 2026-02-24T18:20:31.622 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:31.622 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:31.622 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:31.622 INFO:teuthology.orchestra.run.trial018.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:31.622 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:31.623 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:31.651 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:20:31.666 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:31.714 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:31.732 INFO:teuthology.orchestra.run.trial032.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:31.751 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:31.752 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:31.752 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:31.773 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-cephfs 2026-02-24T18:20:31.779 INFO:teuthology.orchestra.run.trial112.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:31.797 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:31.798 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:20:31.798 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:31.819 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-fuse 2026-02-24T18:20:31.876 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:31.943 INFO:teuthology.orchestra.run.trial110.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:31.962 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:31.962 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:31.962 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:31.968 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:31.985 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librbd1 2026-02-24T18:20:32.052 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:32.053 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:32.053 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:32.053 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:32.053 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:32.053 INFO:teuthology.orchestra.run.trial018.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Installing dependencies: 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Installing weak dependencies: 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Install 3 Packages 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 107 M 2026-02-24T18:20:32.054 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 654 M 2026-02-24T18:20:32.055 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:32.091 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:32.136 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:32.157 INFO:teuthology.orchestra.run.trial032.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:32.177 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:32.178 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:32.178 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:32.204 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rbd 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:32.220 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:32.221 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 85 k 2026-02-24T18:20:32.221 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 227 k 2026-02-24T18:20:32.221 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:32.297 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:32.362 INFO:teuthology.orchestra.run.trial110.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:32.381 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:32.382 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:32.382 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:32.403 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rados 2026-02-24T18:20:32.523 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:32.591 INFO:teuthology.orchestra.run.trial032.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:32.609 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:32.610 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:20:32.610 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:32.613 INFO:teuthology.orchestra.run.trial112.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 217 kB/s | 85 kB 00:00 2026-02-24T18:20:32.615 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:32.615 INFO:teuthology.orchestra.run.trial112.stdout:Total 216 kB/s | 85 kB 00:00 2026-02-24T18:20:32.615 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:32.619 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:32.619 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:32.631 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-fuse 2026-02-24T18:20:32.650 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:32.650 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:32.701 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:32.717 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:32.719 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:32.782 INFO:teuthology.orchestra.run.trial110.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:32.801 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:32.802 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:32.802 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:32.812 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:32.823 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rgw 2026-02-24T18:20:32.875 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:32.876 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:32.876 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:32.876 INFO:teuthology.orchestra.run.trial112.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:32.876 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:32.876 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:32.904 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-mirror 2026-02-24T18:20:32.945 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 85 k 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 227 k 2026-02-24T18:20:33.029 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:33.140 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:33.207 INFO:teuthology.orchestra.run.trial110.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:33.221 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:57 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:33.226 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:33.226 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:33.226 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:33.248 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-cephfs 2026-02-24T18:20:33.306 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:33.306 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:33.306 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:20:33.306 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 3.1 M 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 13 M 2026-02-24T18:20:33.307 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:33.423 INFO:teuthology.orchestra.run.trial032.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 217 kB/s | 85 kB 00:00 2026-02-24T18:20:33.423 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:33.423 INFO:teuthology.orchestra.run.trial032.stdout:Total 217 kB/s | 85 kB 00:00 2026-02-24T18:20:33.424 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:33.427 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:33.427 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:33.457 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:33.457 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:33.509 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:33.526 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:33.567 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:33.612 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:33.633 INFO:teuthology.orchestra.run.trial110.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:33.652 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:33.652 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:33.652 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:33.674 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rbd 2026-02-24T18:20:33.675 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:33.675 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:33.676 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:33.676 INFO:teuthology.orchestra.run.trial032.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:33.676 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:33.676 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:33.702 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-mirror 2026-02-24T18:20:33.986 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:34.021 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:54 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:34.051 INFO:teuthology.orchestra.run.trial110.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:20:34.069 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:34.070 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:20:34.070 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:34.092 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-fuse 2026-02-24T18:20:34.105 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:34.106 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:34.106 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 3.1 M 2026-02-24T18:20:34.107 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 13 M 2026-02-24T18:20:34.108 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:34.190 INFO:teuthology.orchestra.run.trial112.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 3.5 MB/s | 3.1 MB 00:00 2026-02-24T18:20:34.191 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:34.191 INFO:teuthology.orchestra.run.trial112.stdout:Total 3.5 MB/s | 3.1 MB 00:00 2026-02-24T18:20:34.191 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:34.194 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:34.195 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:34.234 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:34.234 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:34.403 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:57 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:34.425 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:34.427 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:34.449 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:34.450 INFO:teuthology.orchestra.run.trial112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:34.450 INFO:teuthology.orchestra.run.trial112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T18:20:34.450 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:34.450 INFO:teuthology.orchestra.run.trial112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:34.450 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:34.489 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:34.489 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:34.489 INFO:teuthology.orchestra.run.trial110.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 85 k 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 227 k 2026-02-24T18:20:34.490 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:34.749 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:34.750 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:34.750 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:34.750 INFO:teuthology.orchestra.run.trial112.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:34.750 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:34.750 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:34.777 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-nbd 2026-02-24T18:20:34.882 INFO:teuthology.orchestra.run.trial110.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 217 kB/s | 85 kB 00:00 2026-02-24T18:20:34.883 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:34.883 INFO:teuthology.orchestra.run.trial110.stdout:Total 216 kB/s | 85 kB 00:00 2026-02-24T18:20:34.883 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:34.886 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:34.887 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:34.917 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:34.917 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:34.958 INFO:teuthology.orchestra.run.trial032.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 3.7 MB/s | 3.1 MB 00:00 2026-02-24T18:20:34.959 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:34.959 INFO:teuthology.orchestra.run.trial032.stdout:Total 3.7 MB/s | 3.1 MB 00:00 2026-02-24T18:20:34.959 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:34.963 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:34.963 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:34.969 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:34.987 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.003 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:35.003 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:35.068 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.091 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:00:59 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:35.130 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:35.157 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-mirror 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:20:35.179 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:35.180 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 170 k 2026-02-24T18:20:35.180 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 490 k 2026-02-24T18:20:35.180 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:35.196 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:35.198 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.221 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.221 INFO:teuthology.orchestra.run.trial032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:35.221 INFO:teuthology.orchestra.run.trial032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T18:20:35.222 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:35.222 INFO:teuthology.orchestra.run.trial032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:35.222 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:35.473 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:00:58 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:35.518 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:35.519 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:35.519 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:35.519 INFO:teuthology.orchestra.run.trial032.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:35.519 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:35.519 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:35.547 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-nbd 2026-02-24T18:20:35.558 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:35.559 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:35.559 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 3.1 M 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 13 M 2026-02-24T18:20:35.560 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:35.863 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:35.922 INFO:teuthology.orchestra.run.trial018.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 2.8 MB/s | 11 MB 00:03 2026-02-24T18:20:35.950 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:35.951 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:35.952 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:20:35.952 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:35.952 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 170 k 2026-02-24T18:20:35.952 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 490 k 2026-02-24T18:20:35.952 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:36.014 INFO:teuthology.orchestra.run.trial112.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 204 kB/s | 170 kB 00:00 2026-02-24T18:20:36.015 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:36.015 INFO:teuthology.orchestra.run.trial112.stdout:Total 204 kB/s | 170 kB 00:00 2026-02-24T18:20:36.015 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:36.019 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:36.019 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:36.050 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:36.050 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:36.107 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:36.124 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.213 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.275 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.275 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:36.275 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:20:36.276 INFO:teuthology.orchestra.run.trial112.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:36.276 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:36.276 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:20:36.305 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:20:36.391 INFO:teuthology.orchestra.run.trial110.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 3.7 MB/s | 3.1 MB 00:00 2026-02-24T18:20:36.392 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:36.392 INFO:teuthology.orchestra.run.trial110.stdout:Total 3.7 MB/s | 3.1 MB 00:00 2026-02-24T18:20:36.392 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:36.396 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:36.396 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:36.436 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:36.436 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:36.479 INFO:teuthology.orchestra.run.trial032.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 323 kB/s | 170 kB 00:00 2026-02-24T18:20:36.480 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:36.480 INFO:teuthology.orchestra.run.trial032.stdout:Total 322 kB/s | 170 kB 00:00 2026-02-24T18:20:36.480 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:20:36.483 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:20:36.483 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:20:36.514 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:20:36.515 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:20:36.573 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:20:36.591 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.626 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:01:00 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:20:36.627 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:36.629 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:20:36.653 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:36.683 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.712 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:20:36.713 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:36.713 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:20:36.713 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:Installing dependencies: 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:Installing weak dependencies: 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:20:36.714 INFO:teuthology.orchestra.run.trial112.stdout:Install 3 Packages 2026-02-24T18:20:36.715 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:20:36.715 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 107 M 2026-02-24T18:20:36.715 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 654 M 2026-02-24T18:20:36.715 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:36.748 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:20:36.780 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:20:36.976 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:36.977 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:36.977 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:36.977 INFO:teuthology.orchestra.run.trial110.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:36.977 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:36.977 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:37.004 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-nbd 2026-02-24T18:20:37.101 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:00:57 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:20:37.186 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:20:37.186 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:37.186 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:20:37.186 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout:Installing dependencies: 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout:Installing weak dependencies: 2026-02-24T18:20:37.187 INFO:teuthology.orchestra.run.trial032.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout:Install 3 Packages 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:20:37.188 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 107 M 2026-02-24T18:20:37.189 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 654 M 2026-02-24T18:20:37.189 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:20:37.327 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:01:00 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:37.412 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:37.412 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:37.412 INFO:teuthology.orchestra.run.trial110.stdout: Package Architecture Version Repository Size 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 170 k 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 490 k 2026-02-24T18:20:37.413 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:37.883 INFO:teuthology.orchestra.run.trial110.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 363 kB/s | 170 kB 00:00 2026-02-24T18:20:37.883 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:37.883 INFO:teuthology.orchestra.run.trial110.stdout:Total 362 kB/s | 170 kB 00:00 2026-02-24T18:20:37.884 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:20:37.887 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:20:37.887 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:20:37.918 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:20:37.918 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:20:37.974 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:20:37.991 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:38.076 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:38.140 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:20:38.170 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:20:38.489 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:01:01 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:20:38.575 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-24T18:20:38.576 INFO:teuthology.orchestra.run.trial110.stdout:Installing dependencies: 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Installing weak dependencies: 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Install 3 Packages 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 107 M 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 654 M 2026-02-24T18:20:38.577 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:20:39.368 INFO:teuthology.orchestra.run.trial112.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 4.1 MB/s | 11 MB 00:02 2026-02-24T18:20:42.940 INFO:teuthology.orchestra.run.trial112.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 3.9 MB/s | 24 MB 00:06 2026-02-24T18:20:44.021 INFO:teuthology.orchestra.run.trial032.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 1.6 MB/s | 11 MB 00:06 2026-02-24T18:20:44.288 INFO:teuthology.orchestra.run.trial018.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 2.0 MB/s | 24 MB 00:12 2026-02-24T18:20:44.547 INFO:teuthology.orchestra.run.trial110.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 1.8 MB/s | 11 MB 00:05 2026-02-24T18:20:48.181 INFO:teuthology.orchestra.run.trial110.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 2.5 MB/s | 24 MB 00:09 2026-02-24T18:20:49.249 INFO:teuthology.orchestra.run.trial018.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 4.2 MB/s | 72 MB 00:17 2026-02-24T18:20:49.251 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:49.251 INFO:teuthology.orchestra.run.trial018.stdout:Total 6.2 MB/s | 107 MB 00:17 2026-02-24T18:20:49.251 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:20:49.256 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:20:49.256 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:20:49.617 INFO:teuthology.orchestra.run.trial032.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 2.0 MB/s | 24 MB 00:12 2026-02-24T18:20:49.635 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:20:49.636 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:20:52.115 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:20:53.383 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-24T18:20:54.515 INFO:teuthology.orchestra.run.trial112.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 4.0 MB/s | 72 MB 00:17 2026-02-24T18:20:54.517 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:54.517 INFO:teuthology.orchestra.run.trial112.stdout:Total 6.0 MB/s | 107 MB 00:17 2026-02-24T18:20:54.517 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:20:54.522 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:20:54.522 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:20:54.902 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:20:54.902 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:20:57.067 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:20:57.086 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:20:57.376 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:20:57.635 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:20:57.635 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-24T18:20:57.635 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:20:57.698 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:57.699 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:20:57.739 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-common-debuginfo 2026-02-24T18:20:58.059 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:01:23 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:20:58.143 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:20:58.143 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:58.143 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:20:58.143 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 474 M 2026-02-24T18:20:58.144 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 2.0 G 2026-02-24T18:20:58.145 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:20:58.667 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-24T18:21:02.427 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:02.447 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:03.001 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:03.001 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-24T18:21:03.002 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:21:03.064 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:21:03.103 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-common-debuginfo 2026-02-24T18:21:03.426 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:21:03.510 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:21:03.510 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:21:03.510 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 474 M 2026-02-24T18:21:03.511 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 2.0 G 2026-02-24T18:21:03.512 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:21:07.683 INFO:teuthology.orchestra.run.trial110.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 2.5 MB/s | 72 MB 00:29 2026-02-24T18:21:07.684 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:21:07.684 INFO:teuthology.orchestra.run.trial110.stdout:Total 3.7 MB/s | 107 MB 00:29 2026-02-24T18:21:07.685 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:21:07.690 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:21:07.690 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:21:08.008 INFO:teuthology.orchestra.run.trial032.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 2.3 MB/s | 72 MB 00:30 2026-02-24T18:21:08.009 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:21:08.010 INFO:teuthology.orchestra.run.trial032.stdout:Total 3.5 MB/s | 107 MB 00:30 2026-02-24T18:21:08.010 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:21:08.014 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:21:08.015 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:21:08.072 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:21:08.072 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:21:08.395 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:21:08.396 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:21:10.553 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:21:10.865 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:21:11.843 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-24T18:21:12.133 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-24T18:21:15.599 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:15.619 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:15.822 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:15.842 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:16.129 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:16.130 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-24T18:21:16.130 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:16.192 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-24T18:21:16.192 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:21:16.193 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:21:16.233 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-common-debuginfo 2026-02-24T18:21:16.345 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:21:16.346 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-24T18:21:16.346 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:21:16.409 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:21:16.410 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:21:16.449 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-common-debuginfo 2026-02-24T18:21:16.547 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:01:39 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:21:16.631 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 474 M 2026-02-24T18:21:16.632 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 2.0 G 2026-02-24T18:21:16.633 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:21:16.772 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:01:36 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:21:16.857 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 474 M 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 2.0 G 2026-02-24T18:21:16.858 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:22:24.878 INFO:teuthology.orchestra.run.trial110.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 6.9 MB/s | 474 MB 01:08 2026-02-24T18:22:24.878 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:22:24.879 INFO:teuthology.orchestra.run.trial110.stdout:Total 6.9 MB/s | 474 MB 01:08 2026-02-24T18:22:24.879 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:22:24.881 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:22:24.881 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:22:26.292 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:22:26.293 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:22:46.144 INFO:teuthology.orchestra.run.trial018.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 4.4 MB/s | 474 MB 01:47 2026-02-24T18:22:46.145 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:22:46.145 INFO:teuthology.orchestra.run.trial018.stdout:Total 4.4 MB/s | 474 MB 01:48 2026-02-24T18:22:46.145 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:22:46.147 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:22:46.147 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:22:47.555 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:22:47.556 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:22:49.369 INFO:teuthology.orchestra.run.trial112.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 4.5 MB/s | 474 MB 01:45 2026-02-24T18:22:49.369 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:22:49.369 INFO:teuthology.orchestra.run.trial112.stdout:Total 4.5 MB/s | 474 MB 01:45 2026-02-24T18:22:49.369 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:22:49.371 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:22:49.371 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:22:50.324 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:22:50.342 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:22:50.782 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:22:50.783 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:22:51.764 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:51.827 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:22:51.895 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-24T18:22:52.213 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:03:15 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:22:52.296 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:22:52.296 INFO:teuthology.orchestra.run.trial110.stdout:======================================================================================= 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:======================================================================================= 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:======================================================================================= 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 1.5 M 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 7.6 M 2026-02-24T18:22:52.297 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:22:52.940 INFO:teuthology.orchestra.run.trial110.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 2.4 MB/s | 1.5 MB 00:00 2026-02-24T18:22:52.940 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:22:52.940 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.4 MB/s | 1.5 MB 00:00 2026-02-24T18:22:52.940 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:22:52.942 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:22:52.942 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:22:52.964 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:22:52.965 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:22:53.079 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:22:53.096 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:22:53.160 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:53.223 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:22:53.250 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-24T18:22:53.573 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:03:16 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-24T18:22:53.658 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 200 M 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 893 M 2026-02-24T18:22:53.659 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:23:11.031 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:23:11.053 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:12.518 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:12.582 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:23:12.652 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-24T18:23:12.974 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:03:37 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:23:13.058 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:23:13.058 INFO:teuthology.orchestra.run.trial018.stdout:======================================================================================= 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:======================================================================================= 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:======================================================================================= 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 1.5 M 2026-02-24T18:23:13.059 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 7.6 M 2026-02-24T18:23:13.060 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:23:14.286 INFO:teuthology.orchestra.run.trial018.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 1.3 MB/s | 1.5 MB 00:01 2026-02-24T18:23:14.286 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:14.286 INFO:teuthology.orchestra.run.trial018.stdout:Total 1.2 MB/s | 1.5 MB 00:01 2026-02-24T18:23:14.286 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:23:14.288 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:23:14.288 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:23:14.311 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:23:14.311 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:23:14.423 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:23:14.441 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:14.504 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:14.568 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:14.568 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:14.568 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:23:14.568 INFO:teuthology.orchestra.run.trial018.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:14.568 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:14.569 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:23:14.596 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-24T18:23:14.704 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:23:14.724 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:14.914 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:03:39 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:23:14.997 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:23:14.997 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:23:14.997 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 200 M 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 893 M 2026-02-24T18:23:14.998 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:23:16.261 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:16.323 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:23:16.392 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-24T18:23:16.708 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:03:40 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:======================================================================================= 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:======================================================================================= 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:======================================================================================= 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 1.5 M 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 7.6 M 2026-02-24T18:23:16.794 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:23:17.748 INFO:teuthology.orchestra.run.trial112.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 1.6 MB/s | 1.5 MB 00:00 2026-02-24T18:23:17.748 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:17.748 INFO:teuthology.orchestra.run.trial112.stdout:Total 1.6 MB/s | 1.5 MB 00:00 2026-02-24T18:23:17.748 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:23:17.750 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:23:17.750 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:23:17.772 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:23:17.772 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:23:17.885 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:23:17.903 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:17.967 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:18.031 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:23:18.059 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-24T18:23:18.374 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:03:42 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:23:18.458 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:23:18.458 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:23:18.458 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:23:18.458 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 200 M 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 893 M 2026-02-24T18:23:18.459 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:23:29.681 INFO:teuthology.orchestra.run.trial032.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 3.6 MB/s | 474 MB 02:12 2026-02-24T18:23:29.681 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:29.681 INFO:teuthology.orchestra.run.trial032.stdout:Total 3.6 MB/s | 474 MB 02:12 2026-02-24T18:23:29.682 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:23:29.684 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:23:29.684 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:23:31.094 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:23:31.094 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:23:39.989 INFO:teuthology.orchestra.run.trial110.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 4.3 MB/s | 200 MB 00:46 2026-02-24T18:23:39.990 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:39.990 INFO:teuthology.orchestra.run.trial110.stdout:Total 4.3 MB/s | 200 MB 00:46 2026-02-24T18:23:39.990 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:23:39.992 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:23:39.992 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:23:40.598 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:23:40.598 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:23:50.859 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:23:50.878 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:23:51.570 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:23:51.633 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:23:51.633 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:23:51.633 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:23:51.634 INFO:teuthology.orchestra.run.trial110.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:51.634 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:23:51.634 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:23:51.677 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-test-debuginfo 2026-02-24T18:23:51.993 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:04:14 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:23:52.077 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 729 M 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 3.6 G 2026-02-24T18:23:52.078 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:23:54.617 INFO:teuthology.orchestra.run.trial112.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 5.5 MB/s | 200 MB 00:36 2026-02-24T18:23:54.618 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:54.618 INFO:teuthology.orchestra.run.trial112.stdout:Total 5.5 MB/s | 200 MB 00:36 2026-02-24T18:23:54.618 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:23:54.620 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:23:54.620 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:23:54.637 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:23:54.657 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:55.228 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:23:55.228 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:23:56.185 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:56.248 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:23:56.321 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-24T18:23:56.642 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:04:16 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout:======================================================================================= 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout:======================================================================================= 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-24T18:23:56.727 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:======================================================================================= 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 1.5 M 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 7.6 M 2026-02-24T18:23:56.728 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:23:57.518 INFO:teuthology.orchestra.run.trial032.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 1.9 MB/s | 1.5 MB 00:00 2026-02-24T18:23:57.518 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:23:57.519 INFO:teuthology.orchestra.run.trial032.stdout:Total 1.9 MB/s | 1.5 MB 00:00 2026-02-24T18:23:57.519 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:23:57.520 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:23:57.520 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:23:57.542 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:23:57.542 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:23:57.656 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:23:57.674 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:57.746 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:57.811 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:23:57.811 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:57.812 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:23:57.812 INFO:teuthology.orchestra.run.trial032.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:23:57.812 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:57.812 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:23:57.839 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-24T18:23:58.160 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:04:18 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:58.244 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 200 M 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 893 M 2026-02-24T18:23:58.245 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:24:05.275 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:24:05.293 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:05.995 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:06.058 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:06.059 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:24:06.059 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:24:06.059 INFO:teuthology.orchestra.run.trial112.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:24:06.059 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:24:06.059 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:24:06.103 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-test-debuginfo 2026-02-24T18:24:06.427 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:04:30 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:24:06.513 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:24:06.513 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:24:06.514 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 729 M 2026-02-24T18:24:06.515 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 3.6 G 2026-02-24T18:24:06.515 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:24:29.042 INFO:teuthology.orchestra.run.trial018.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 2.7 MB/s | 200 MB 01:14 2026-02-24T18:24:29.043 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:24:29.043 INFO:teuthology.orchestra.run.trial018.stdout:Total 2.7 MB/s | 200 MB 01:14 2026-02-24T18:24:29.043 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:24:29.045 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:24:29.045 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:24:29.650 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:24:29.651 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:24:35.650 INFO:teuthology.orchestra.run.trial032.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 5.4 MB/s | 200 MB 00:37 2026-02-24T18:24:35.650 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:24:35.650 INFO:teuthology.orchestra.run.trial032.stdout:Total 5.4 MB/s | 200 MB 00:37 2026-02-24T18:24:35.651 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:24:35.653 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:24:35.653 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:24:36.259 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:24:36.259 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:24:39.691 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:24:39.710 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:40.411 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:40.473 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:40.474 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:24:40.474 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:24:40.474 INFO:teuthology.orchestra.run.trial018.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:24:40.474 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:24:40.474 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:24:40.516 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-test-debuginfo 2026-02-24T18:24:40.827 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:05:05 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:24:40.910 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 729 M 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 3.6 G 2026-02-24T18:24:40.911 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:24:46.300 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:24:46.318 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:46.999 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:24:47.063 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:24:47.107 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-test-debuginfo 2026-02-24T18:24:47.424 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:05:07 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:24:47.507 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:24:47.508 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:24:47.509 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 729 M 2026-02-24T18:24:47.509 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 3.6 G 2026-02-24T18:24:47.509 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:26:13.022 INFO:teuthology.orchestra.run.trial112.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 5.8 MB/s | 729 MB 02:06 2026-02-24T18:26:13.022 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:26:13.022 INFO:teuthology.orchestra.run.trial112.stdout:Total 5.8 MB/s | 729 MB 02:06 2026-02-24T18:26:13.022 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:26:13.025 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:26:13.025 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:26:15.197 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:26:15.197 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:26:20.699 INFO:teuthology.orchestra.run.trial032.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 7.8 MB/s | 729 MB 01:33 2026-02-24T18:26:20.699 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:26:20.700 INFO:teuthology.orchestra.run.trial032.stdout:Total 7.8 MB/s | 729 MB 01:33 2026-02-24T18:26:20.700 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:26:20.703 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:26:20.703 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:26:22.859 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:26:22.859 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:26:42.029 INFO:teuthology.orchestra.run.trial110.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 4.3 MB/s | 729 MB 02:49 2026-02-24T18:26:42.030 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:26:42.030 INFO:teuthology.orchestra.run.trial110.stdout:Total 4.3 MB/s | 729 MB 02:49 2026-02-24T18:26:42.030 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:26:42.033 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:26:42.033 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:26:44.197 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:26:44.197 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:26:53.650 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:26:53.670 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:54.652 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:26:54.716 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:26:54.818 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:26:55.142 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:07:19 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:26:55.208 INFO:teuthology.orchestra.run.trial112.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:26:55.225 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:26:55.226 INFO:teuthology.orchestra.run.trial112.stdout:Nothing to do. 2026-02-24T18:26:55.226 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:26:55.248 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mgr-debuginfo 2026-02-24T18:26:55.569 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:07:19 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:26:55.653 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:26:55.654 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:26:55.655 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:26:55.655 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 18 M 2026-02-24T18:26:55.655 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 110 M 2026-02-24T18:26:55.655 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:27:00.554 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:27:00.577 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:01.163 INFO:teuthology.orchestra.run.trial112.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 3.3 MB/s | 18 MB 00:05 2026-02-24T18:27:01.163 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:01.163 INFO:teuthology.orchestra.run.trial112.stdout:Total 3.3 MB/s | 18 MB 00:05 2026-02-24T18:27:01.163 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:27:01.165 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:27:01.165 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:27:01.236 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:27:01.236 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:27:02.251 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:27:02.270 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:02.412 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:02.475 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:27:02.505 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mds-debuginfo 2026-02-24T18:27:02.829 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:07:26 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:27:02.914 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:27:02.914 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 55 M 2026-02-24T18:27:02.915 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 211 M 2026-02-24T18:27:02.916 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:27:03.034 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:03.098 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:27:03.197 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:27:03.517 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:07:23 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:27:03.583 INFO:teuthology.orchestra.run.trial032.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:27:03.601 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:27:03.602 INFO:teuthology.orchestra.run.trial032.stdout:Nothing to do. 2026-02-24T18:27:03.602 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:27:03.622 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mgr-debuginfo 2026-02-24T18:27:03.945 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:07:23 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:27:04.028 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 18 M 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 110 M 2026-02-24T18:27:04.029 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:27:06.501 INFO:teuthology.orchestra.run.trial032.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 7.3 MB/s | 18 MB 00:02 2026-02-24T18:27:06.501 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:06.501 INFO:teuthology.orchestra.run.trial032.stdout:Total 7.3 MB/s | 18 MB 00:02 2026-02-24T18:27:06.501 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:27:06.503 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:27:06.503 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:27:06.575 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:27:06.575 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:27:06.717 INFO:teuthology.orchestra.run.trial112.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 15 MB/s | 55 MB 00:03 2026-02-24T18:27:06.718 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:06.718 INFO:teuthology.orchestra.run.trial112.stdout:Total 15 MB/s | 55 MB 00:03 2026-02-24T18:27:06.718 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:27:06.720 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:27:06.720 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:27:06.902 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:27:06.902 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:27:06.943 INFO:teuthology.orchestra.run.trial018.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 5.0 MB/s | 729 MB 02:26 2026-02-24T18:27:06.944 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:06.944 INFO:teuthology.orchestra.run.trial018.stdout:Total 5.0 MB/s | 729 MB 02:26 2026-02-24T18:27:06.944 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:27:06.947 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:27:06.947 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:27:07.612 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:27:07.631 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:07.765 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:07.829 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:07.829 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:07.829 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:27:07.830 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:07.830 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:07.830 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:27:07.859 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mds-debuginfo 2026-02-24T18:27:08.181 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:07:28 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:27:08.266 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:27:08.266 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:08.267 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 55 M 2026-02-24T18:27:08.268 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 211 M 2026-02-24T18:27:08.268 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:27:09.106 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:27:09.107 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:27:09.645 INFO:teuthology.orchestra.run.trial032.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 40 MB/s | 55 MB 00:01 2026-02-24T18:27:09.646 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:09.646 INFO:teuthology.orchestra.run.trial032.stdout:Total 40 MB/s | 55 MB 00:01 2026-02-24T18:27:09.646 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:27:09.648 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:27:09.648 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:27:09.675 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:27:09.695 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:09.827 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:27:09.828 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:27:09.924 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:09.986 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:09.987 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:09.987 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:27:09.987 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:09.987 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:09.987 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:27:10.018 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-mon-debuginfo 2026-02-24T18:27:10.337 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:07:34 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:27:10.420 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:27:10.421 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 102 M 2026-02-24T18:27:10.421 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 497 M 2026-02-24T18:27:10.421 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:27:12.547 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:27:12.565 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:12.785 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:12.848 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:27:12.879 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-mon-debuginfo 2026-02-24T18:27:13.196 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:07:33 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:27:13.278 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:27:13.278 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:13.278 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:27:13.278 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 102 M 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 497 M 2026-02-24T18:27:13.279 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:27:18.670 INFO:teuthology.orchestra.run.trial032.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 19 MB/s | 102 MB 00:05 2026-02-24T18:27:18.670 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:18.671 INFO:teuthology.orchestra.run.trial032.stdout:Total 19 MB/s | 102 MB 00:05 2026-02-24T18:27:18.671 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:27:18.673 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:27:18.673 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:27:18.987 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:27:18.988 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:27:23.172 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:27:23.193 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:24.376 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:27:24.395 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:24.792 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:24.856 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:24.857 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:24.857 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:27:24.857 INFO:teuthology.orchestra.run.trial032.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:24.857 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:24.857 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:27:24.894 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-osd-debuginfo 2026-02-24T18:27:25.189 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:25.215 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:07:45 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:25.252 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 298 M 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 1.4 G 2026-02-24T18:27:25.299 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:27:25.350 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:27:25.679 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:07:48 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:27:25.745 INFO:teuthology.orchestra.run.trial110.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:27:25.763 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:27:25.764 INFO:teuthology.orchestra.run.trial110.stdout:Nothing to do. 2026-02-24T18:27:25.764 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:27:25.785 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mgr-debuginfo 2026-02-24T18:27:26.104 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:07:49 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-24T18:27:26.187 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 18 M 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 110 M 2026-02-24T18:27:26.188 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:27:34.879 INFO:teuthology.orchestra.run.trial110.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 2.1 MB/s | 18 MB 00:08 2026-02-24T18:27:34.879 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:34.880 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.1 MB/s | 18 MB 00:08 2026-02-24T18:27:34.880 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:27:34.881 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:27:34.881 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:27:34.952 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:27:34.952 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:27:35.971 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:27:35.989 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:36.125 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:36.188 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:36.188 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:36.188 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:27:36.188 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:36.188 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:36.189 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:27:36.219 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mds-debuginfo 2026-02-24T18:27:36.541 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:07:59 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:27:36.627 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 55 M 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 211 M 2026-02-24T18:27:36.628 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:27:39.261 INFO:teuthology.orchestra.run.trial110.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 21 MB/s | 55 MB 00:02 2026-02-24T18:27:39.262 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:39.262 INFO:teuthology.orchestra.run.trial110.stdout:Total 21 MB/s | 55 MB 00:02 2026-02-24T18:27:39.262 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:27:39.264 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:27:39.264 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:27:39.444 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:27:39.444 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:27:42.231 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:27:42.250 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:42.474 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:42.538 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:42.539 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:42.539 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:27:42.539 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:42.539 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:42.539 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:27:42.575 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-mon-debuginfo 2026-02-24T18:27:42.896 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:08:05 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:27:42.981 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:42.982 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 102 M 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 497 M 2026-02-24T18:27:42.983 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:27:47.747 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:27:47.768 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:48.890 INFO:teuthology.orchestra.run.trial110.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 17 MB/s | 102 MB 00:05 2026-02-24T18:27:48.891 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:48.891 INFO:teuthology.orchestra.run.trial110.stdout:Total 17 MB/s | 102 MB 00:05 2026-02-24T18:27:48.891 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:27:48.893 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:27:48.893 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:27:49.209 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:27:49.209 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:27:49.357 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:49.419 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:27:49.419 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:49.420 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:27:49.420 INFO:teuthology.orchestra.run.trial018.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:49.420 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:49.420 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:27:49.513 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:27:49.833 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:08:14 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:27:49.899 INFO:teuthology.orchestra.run.trial018.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:27:49.917 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:27:49.917 INFO:teuthology.orchestra.run.trial018.stdout:Nothing to do. 2026-02-24T18:27:49.917 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:27:49.939 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mgr-debuginfo 2026-02-24T18:27:50.261 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:08:15 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:27:50.347 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:27:50.347 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:50.347 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:27:50.347 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 18 M 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 110 M 2026-02-24T18:27:50.348 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:27:53.259 INFO:teuthology.orchestra.run.trial018.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 6.2 MB/s | 18 MB 00:02 2026-02-24T18:27:53.259 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:53.259 INFO:teuthology.orchestra.run.trial018.stdout:Total 6.2 MB/s | 18 MB 00:02 2026-02-24T18:27:53.259 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:27:53.260 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:27:53.261 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:27:53.331 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:27:53.331 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:27:54.231 INFO:teuthology.orchestra.run.trial112.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 2.3 MB/s | 102 MB 00:43 2026-02-24T18:27:54.232 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:54.232 INFO:teuthology.orchestra.run.trial112.stdout:Total 2.3 MB/s | 102 MB 00:43 2026-02-24T18:27:54.232 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:27:54.234 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:27:54.234 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:27:54.343 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:27:54.362 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:54.505 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:54.529 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:27:54.547 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:54.548 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:27:54.548 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:27:54.569 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:54.569 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:54.569 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:27:54.569 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:54.569 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:54.570 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:27:54.600 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mds-debuginfo 2026-02-24T18:27:54.921 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:08:19 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:27:54.953 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:55.004 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:55.005 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 55 M 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 211 M 2026-02-24T18:27:55.006 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:55.017 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:27:55.055 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-osd-debuginfo 2026-02-24T18:27:55.376 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:08:18 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:27:55.461 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 298 M 2026-02-24T18:27:55.462 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 1.4 G 2026-02-24T18:27:55.462 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:27:59.859 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:27:59.877 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:00.258 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:00.321 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:00.321 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:28:00.321 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:28:00.321 INFO:teuthology.orchestra.run.trial112.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:00.322 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:28:00.322 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:28:00.356 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-osd-debuginfo 2026-02-24T18:28:00.675 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:08:24 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:28:00.759 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 298 M 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 1.4 G 2026-02-24T18:28:00.760 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:28:09.828 INFO:teuthology.orchestra.run.trial018.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 3.7 MB/s | 55 MB 00:14 2026-02-24T18:28:09.829 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:09.829 INFO:teuthology.orchestra.run.trial018.stdout:Total 3.7 MB/s | 55 MB 00:14 2026-02-24T18:28:09.829 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:28:09.830 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:28:09.831 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:28:10.011 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:28:10.012 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:28:12.691 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:28:12.709 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:12.903 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:12.965 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:12.965 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:12.966 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:28:12.966 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:12.966 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:12.966 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:28:12.999 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-mon-debuginfo 2026-02-24T18:28:13.318 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:08:38 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:28:13.402 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:28:13.402 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:13.402 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 102 M 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 497 M 2026-02-24T18:28:13.403 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:28:20.406 INFO:teuthology.orchestra.run.trial032.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 5.4 MB/s | 298 MB 00:55 2026-02-24T18:28:20.407 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:20.407 INFO:teuthology.orchestra.run.trial032.stdout:Total 5.4 MB/s | 298 MB 00:55 2026-02-24T18:28:20.407 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:28:20.409 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:28:20.409 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:28:21.302 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:28:21.302 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:28:36.376 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:28:36.395 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:37.416 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:37.481 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:28:37.534 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install ceph-fuse-debuginfo 2026-02-24T18:28:37.853 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:08:57 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:28:37.936 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:28:37.936 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:37.936 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:28:37.936 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 6.8 M 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 36 M 2026-02-24T18:28:37.937 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:28:40.442 INFO:teuthology.orchestra.run.trial032.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 2.7 MB/s | 6.8 MB 00:02 2026-02-24T18:28:40.443 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:40.443 INFO:teuthology.orchestra.run.trial032.stdout:Total 2.7 MB/s | 6.8 MB 00:02 2026-02-24T18:28:40.443 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:28:40.445 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:28:40.445 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:28:40.482 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:28:40.483 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:28:40.875 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:28:40.893 INFO:teuthology.orchestra.run.trial032.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:28:40.979 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:28:41.043 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:28:41.043 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:41.043 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:28:41.044 INFO:teuthology.orchestra.run.trial032.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:41.044 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:41.044 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:28:41.072 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librados-devel-debuginfo 2026-02-24T18:28:41.390 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:01 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:28:41.475 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:28:41.476 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:41.476 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:28:41.476 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:41.476 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 112 k 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 493 k 2026-02-24T18:28:41.477 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:28:41.777 INFO:teuthology.orchestra.run.trial018.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 3.6 MB/s | 102 MB 00:28 2026-02-24T18:28:41.778 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:41.778 INFO:teuthology.orchestra.run.trial018.stdout:Total 3.6 MB/s | 102 MB 00:28 2026-02-24T18:28:41.778 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:28:41.779 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:28:41.779 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:28:41.953 INFO:teuthology.orchestra.run.trial032.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 235 kB/s | 112 kB 00:00 2026-02-24T18:28:41.954 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:41.954 INFO:teuthology.orchestra.run.trial032.stdout:Total 234 kB/s | 112 kB 00:00 2026-02-24T18:28:41.955 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:28:41.956 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:28:41.956 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:28:41.975 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:28:41.976 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:28:42.016 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:28:42.034 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:28:42.094 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:28:42.095 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:28:42.095 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:28:42.159 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:28:42.159 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:42.159 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:28:42.159 INFO:teuthology.orchestra.run.trial032.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:42.160 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:42.160 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:28:42.186 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install libcephfs2-debuginfo 2026-02-24T18:28:42.503 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:02 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:28:42.586 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 6.0 M 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 31 M 2026-02-24T18:28:42.587 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:28:44.373 INFO:teuthology.orchestra.run.trial032.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 3.3 MB/s | 6.0 MB 00:01 2026-02-24T18:28:44.373 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:44.373 INFO:teuthology.orchestra.run.trial032.stdout:Total 3.3 MB/s | 6.0 MB 00:01 2026-02-24T18:28:44.373 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:28:44.375 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:28:44.375 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:28:44.412 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:28:44.412 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:28:44.759 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:28:44.779 INFO:teuthology.orchestra.run.trial032.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:28:44.865 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:28:44.930 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:28:44.930 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:44.930 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:28:44.931 INFO:teuthology.orchestra.run.trial032.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:44.931 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:44.931 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:28:44.959 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librados2-debuginfo 2026-02-24T18:28:45.284 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:05 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-24T18:28:45.368 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 47 M 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 302 M 2026-02-24T18:28:45.369 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:28:47.550 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:28:47.568 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:47.965 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:48.028 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:48.029 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:48.029 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:28:48.029 INFO:teuthology.orchestra.run.trial018.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:48.029 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:48.029 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:28:48.065 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-osd-debuginfo 2026-02-24T18:28:48.381 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:09:13 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:28:48.464 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:28:48.464 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:28:48.465 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 298 M 2026-02-24T18:28:48.466 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 1.4 G 2026-02-24T18:28:48.466 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:28:54.814 INFO:teuthology.orchestra.run.trial112.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 5.5 MB/s | 298 MB 00:54 2026-02-24T18:28:54.814 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:54.815 INFO:teuthology.orchestra.run.trial112.stdout:Total 5.5 MB/s | 298 MB 00:54 2026-02-24T18:28:54.815 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:28:54.817 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:28:54.817 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:28:55.712 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:28:55.712 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:04.520 INFO:teuthology.orchestra.run.trial032.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 2.5 MB/s | 47 MB 00:19 2026-02-24T18:29:04.521 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:04.522 INFO:teuthology.orchestra.run.trial032.stdout:Total 2.5 MB/s | 47 MB 00:19 2026-02-24T18:29:04.522 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:04.524 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:04.524 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:04.682 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:04.682 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:07.329 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:07.348 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:07.625 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:07.690 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:07.724 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install librbd1-debuginfo 2026-02-24T18:29:08.047 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:28 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:08.131 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:08.131 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:08.132 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:08.133 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 55 M 2026-02-24T18:29:08.133 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 374 M 2026-02-24T18:29:08.133 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:10.714 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:29:10.732 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:11.819 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:11.881 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:29:11.934 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install ceph-fuse-debuginfo 2026-02-24T18:29:12.258 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:09:36 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-24T18:29:12.342 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 6.8 M 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 36 M 2026-02-24T18:29:12.343 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:29:12.700 INFO:teuthology.orchestra.run.trial018.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 12 MB/s | 298 MB 00:24 2026-02-24T18:29:12.701 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:12.701 INFO:teuthology.orchestra.run.trial018.stdout:Total 12 MB/s | 298 MB 00:24 2026-02-24T18:29:12.701 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:29:12.703 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:29:12.703 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:29:12.780 INFO:teuthology.orchestra.run.trial112.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 16 MB/s | 6.8 MB 00:00 2026-02-24T18:29:12.781 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:12.781 INFO:teuthology.orchestra.run.trial112.stdout:Total 16 MB/s | 6.8 MB 00:00 2026-02-24T18:29:12.781 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:29:12.783 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:29:12.783 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:29:12.821 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:29:12.821 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:13.215 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:29:13.233 INFO:teuthology.orchestra.run.trial112.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:13.323 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:13.386 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:29:13.414 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librados-devel-debuginfo 2026-02-24T18:29:13.593 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:29:13.594 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:29:13.735 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:09:37 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:29:13.820 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:29:13.821 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:13.822 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 112 k 2026-02-24T18:29:13.823 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 493 k 2026-02-24T18:29:13.823 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:29:14.132 INFO:teuthology.orchestra.run.trial112.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 360 kB/s | 112 kB 00:00 2026-02-24T18:29:14.134 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:14.134 INFO:teuthology.orchestra.run.trial112.stdout:Total 358 kB/s | 112 kB 00:00 2026-02-24T18:29:14.134 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:29:14.136 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:29:14.136 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:29:14.155 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:29:14.155 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:14.194 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:29:14.212 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:14.276 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:14.339 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:29:14.367 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install libcephfs2-debuginfo 2026-02-24T18:29:14.684 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:09:38 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 6.0 M 2026-02-24T18:29:14.769 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 31 M 2026-02-24T18:29:14.770 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:29:17.066 INFO:teuthology.orchestra.run.trial112.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.6 MB/s | 6.0 MB 00:02 2026-02-24T18:29:17.067 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:17.067 INFO:teuthology.orchestra.run.trial112.stdout:Total 2.6 MB/s | 6.0 MB 00:02 2026-02-24T18:29:17.067 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:29:17.069 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:29:17.069 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:29:17.105 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:29:17.105 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:17.464 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:29:17.483 INFO:teuthology.orchestra.run.trial112.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:17.564 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:17.627 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:29:17.655 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librados2-debuginfo 2026-02-24T18:29:17.970 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:09:41 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:29:18.056 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:18.057 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:29:18.058 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:18.058 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 47 M 2026-02-24T18:29:18.058 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 302 M 2026-02-24T18:29:18.058 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:29:26.384 INFO:teuthology.orchestra.run.trial112.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 5.7 MB/s | 47 MB 00:08 2026-02-24T18:29:26.385 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:26.385 INFO:teuthology.orchestra.run.trial112.stdout:Total 5.7 MB/s | 47 MB 00:08 2026-02-24T18:29:26.385 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:29:26.386 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:29:26.387 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:29:26.546 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:29:26.546 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:29.249 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:29:29.268 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:29.356 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:29:29.376 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:29.539 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:29.603 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:29:29.636 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install librbd1-debuginfo 2026-02-24T18:29:29.949 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:09:53 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:29:30.036 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:29:30.036 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:29:30.037 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 55 M 2026-02-24T18:29:30.038 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 374 M 2026-02-24T18:29:30.038 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:29:30.393 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:30.457 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:29:30.513 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install ceph-fuse-debuginfo 2026-02-24T18:29:30.833 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:09:55 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:29:30.917 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-24T18:29:30.918 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 6.8 M 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 36 M 2026-02-24T18:29:30.919 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:29:31.520 INFO:teuthology.orchestra.run.trial032.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 2.3 MB/s | 55 MB 00:23 2026-02-24T18:29:31.521 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:31.521 INFO:teuthology.orchestra.run.trial032.stdout:Total 2.3 MB/s | 55 MB 00:23 2026-02-24T18:29:31.521 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:31.523 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:31.523 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:31.704 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:31.704 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:32.323 INFO:teuthology.orchestra.run.trial018.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 4.9 MB/s | 6.8 MB 00:01 2026-02-24T18:29:32.323 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:32.323 INFO:teuthology.orchestra.run.trial018.stdout:Total 4.9 MB/s | 6.8 MB 00:01 2026-02-24T18:29:32.323 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:29:32.325 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:29:32.325 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:29:32.363 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:29:32.363 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:29:32.756 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:29:32.774 INFO:teuthology.orchestra.run.trial018.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:32.861 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:32.925 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:29:32.953 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librados-devel-debuginfo 2026-02-24T18:29:33.269 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:09:58 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:33.353 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 112 k 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 493 k 2026-02-24T18:29:33.354 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:29:33.820 INFO:teuthology.orchestra.run.trial018.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 239 kB/s | 112 kB 00:00 2026-02-24T18:29:33.822 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:33.822 INFO:teuthology.orchestra.run.trial018.stdout:Total 238 kB/s | 112 kB 00:00 2026-02-24T18:29:33.823 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:29:33.824 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:29:33.825 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:29:33.843 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:29:33.843 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:29:33.882 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:29:33.900 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:33.960 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:34.023 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:29:34.049 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install libcephfs2-debuginfo 2026-02-24T18:29:34.364 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:09:59 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:29:34.449 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:29:34.449 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 6.0 M 2026-02-24T18:29:34.450 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 31 M 2026-02-24T18:29:34.451 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:29:34.894 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:34.913 INFO:teuthology.orchestra.run.trial032.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:35.241 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:35.305 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:35.336 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-cephfs-debuginfo 2026-02-24T18:29:35.655 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:55 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-24T18:29:35.740 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 628 k 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 2.1 M 2026-02-24T18:29:35.741 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:36.102 INFO:teuthology.orchestra.run.trial018.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 3.6 MB/s | 6.0 MB 00:01 2026-02-24T18:29:36.102 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:36.102 INFO:teuthology.orchestra.run.trial018.stdout:Total 3.6 MB/s | 6.0 MB 00:01 2026-02-24T18:29:36.102 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:29:36.104 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:29:36.104 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:29:36.140 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:29:36.140 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:29:36.319 INFO:teuthology.orchestra.run.trial032.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 1.1 MB/s | 628 kB 00:00 2026-02-24T18:29:36.320 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:36.320 INFO:teuthology.orchestra.run.trial032.stdout:Total 1.1 MB/s | 628 kB 00:00 2026-02-24T18:29:36.320 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:36.322 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:36.322 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:36.342 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:36.342 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:36.406 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:36.424 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:36.484 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:36.491 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:29:36.509 INFO:teuthology.orchestra.run.trial018.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:36.548 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:36.549 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:36.549 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:36.549 INFO:teuthology.orchestra.run.trial032.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:36.549 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:36.549 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:36.575 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rados-debuginfo 2026-02-24T18:29:36.592 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:36.656 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:36.656 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:36.657 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:29:36.657 INFO:teuthology.orchestra.run.trial018.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:36.657 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:36.657 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:29:36.685 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librados2-debuginfo 2026-02-24T18:29:36.897 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:56 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:36.987 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 779 k 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 2.8 M 2026-02-24T18:29:36.988 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:37.004 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:10:02 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:29:37.087 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 47 M 2026-02-24T18:29:37.088 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 302 M 2026-02-24T18:29:37.089 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:29:37.436 INFO:teuthology.orchestra.run.trial032.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 1.7 MB/s | 779 kB 00:00 2026-02-24T18:29:37.437 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:37.437 INFO:teuthology.orchestra.run.trial032.stdout:Total 1.7 MB/s | 779 kB 00:00 2026-02-24T18:29:37.438 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:37.439 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:37.439 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:37.460 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:37.460 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:37.534 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:37.552 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:37.614 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:37.679 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:37.679 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:37.679 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:37.679 INFO:teuthology.orchestra.run.trial032.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:37.680 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:37.680 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:37.707 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rbd-debuginfo 2026-02-24T18:29:38.031 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:58 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:38.114 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:38.115 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 1.1 M 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 4.3 M 2026-02-24T18:29:38.116 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:38.711 INFO:teuthology.orchestra.run.trial032.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 1.9 MB/s | 1.1 MB 00:00 2026-02-24T18:29:38.712 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:38.712 INFO:teuthology.orchestra.run.trial032.stdout:Total 1.9 MB/s | 1.1 MB 00:00 2026-02-24T18:29:38.712 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:38.713 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:38.713 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:38.735 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:38.736 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:38.828 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:38.847 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:38.910 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:38.975 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:38.975 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:38.976 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:38.976 INFO:teuthology.orchestra.run.trial032.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:38.976 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:38.976 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:39.004 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install python3-rgw-debuginfo 2026-02-24T18:29:39.324 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:09:59 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:39.411 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:39.412 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:39.412 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 339 k 2026-02-24T18:29:39.413 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 1.0 M 2026-02-24T18:29:39.414 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:39.922 INFO:teuthology.orchestra.run.trial032.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 665 kB/s | 339 kB 00:00 2026-02-24T18:29:39.923 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:39.923 INFO:teuthology.orchestra.run.trial032.stdout:Total 664 kB/s | 339 kB 00:00 2026-02-24T18:29:39.923 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:39.925 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:39.925 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:39.944 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:39.945 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:39.995 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:40.014 INFO:teuthology.orchestra.run.trial032.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:40.076 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:40.140 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:40.167 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-fuse-debuginfo 2026-02-24T18:29:40.495 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:10:00 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:40.581 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:40.581 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:40.582 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 547 k 2026-02-24T18:29:40.583 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 3.0 M 2026-02-24T18:29:40.583 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:41.170 INFO:teuthology.orchestra.run.trial032.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 930 kB/s | 547 kB 00:00 2026-02-24T18:29:41.171 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:41.171 INFO:teuthology.orchestra.run.trial032.stdout:Total 929 kB/s | 547 kB 00:00 2026-02-24T18:29:41.171 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:41.172 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:41.173 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:41.193 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:41.193 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:29:41.257 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:29:41.275 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:41.338 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:41.402 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:41.402 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:41.403 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:29:41.403 INFO:teuthology.orchestra.run.trial032.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:41.403 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:41.403 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:29:41.431 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-mirror-debuginfo 2026-02-24T18:29:41.749 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:10:01 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:29:41.835 INFO:teuthology.orchestra.run.trial032.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 54 M 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 353 M 2026-02-24T18:29:41.836 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:29:43.894 INFO:teuthology.orchestra.run.trial018.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 6.9 MB/s | 47 MB 00:06 2026-02-24T18:29:43.895 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:43.895 INFO:teuthology.orchestra.run.trial018.stdout:Total 6.9 MB/s | 47 MB 00:06 2026-02-24T18:29:43.895 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:29:43.896 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:29:43.897 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:29:44.055 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:29:44.055 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:29:46.826 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:29:46.844 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:47.124 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:47.187 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:47.187 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:47.187 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:29:47.188 INFO:teuthology.orchestra.run.trial018.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:47.188 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:47.188 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:29:47.218 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install librbd1-debuginfo 2026-02-24T18:29:47.533 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:10:12 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:29:47.616 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:29:47.617 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:47.617 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:29:47.617 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:47.617 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 55 M 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 374 M 2026-02-24T18:29:47.618 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:29:57.798 INFO:teuthology.orchestra.run.trial112.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 2.0 MB/s | 55 MB 00:27 2026-02-24T18:29:57.799 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:57.799 INFO:teuthology.orchestra.run.trial112.stdout:Total 2.0 MB/s | 55 MB 00:27 2026-02-24T18:29:57.799 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:29:57.801 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:29:57.801 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:29:57.982 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:29:57.982 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:29:58.827 INFO:teuthology.orchestra.run.trial032.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 3.2 MB/s | 54 MB 00:16 2026-02-24T18:29:58.828 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:58.828 INFO:teuthology.orchestra.run.trial032.stdout:Total 3.2 MB/s | 54 MB 00:16 2026-02-24T18:29:58.828 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:29:58.829 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:29:58.830 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:29:59.005 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:29:59.005 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:30:01.212 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:01.231 INFO:teuthology.orchestra.run.trial112.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:01.581 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:01.645 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:01.646 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:01.646 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:01.646 INFO:teuthology.orchestra.run.trial112.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:01.646 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:01.646 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:01.678 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-cephfs-debuginfo 2026-02-24T18:30:02.000 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:25 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:02.056 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:30:02.075 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:02.085 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:02.086 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 628 k 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 2.1 M 2026-02-24T18:30:02.087 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:02.360 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:02.424 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:02.425 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:02.425 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:30:02.425 INFO:teuthology.orchestra.run.trial032.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:02.425 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:02.425 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:30:02.458 DEBUG:teuthology.orchestra.run.trial032:> sudo yum -y install rbd-nbd-debuginfo 2026-02-24T18:30:02.771 INFO:teuthology.orchestra.run.trial112.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 919 kB/s | 628 kB 00:00 2026-02-24T18:30:02.771 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:02.771 INFO:teuthology.orchestra.run.trial112.stdout:Total 918 kB/s | 628 kB 00:00 2026-02-24T18:30:02.771 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:02.773 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:02.773 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:02.779 INFO:teuthology.orchestra.run.trial032.stdout:Last metadata expiration check: 0:10:22 ago on Tue 24 Feb 2026 06:19:40 PM UTC. 2026-02-24T18:30:02.792 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:02.793 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:02.857 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:02.862 INFO:teuthology.orchestra.run.trial032.stdout:Dependencies resolved. 2026-02-24T18:30:02.862 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout: Package Arch Version Repo Size 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout:Installing: 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout:Transaction Summary 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout:================================================================================ 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout:Install 1 Package 2026-02-24T18:30:02.863 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:02.864 INFO:teuthology.orchestra.run.trial032.stdout:Total download size: 1.4 M 2026-02-24T18:30:02.864 INFO:teuthology.orchestra.run.trial032.stdout:Installed size: 6.6 M 2026-02-24T18:30:02.864 INFO:teuthology.orchestra.run.trial032.stdout:Downloading Packages: 2026-02-24T18:30:02.874 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:02.936 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:02.998 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:02.999 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:02.999 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:02.999 INFO:teuthology.orchestra.run.trial112.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:02.999 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:02.999 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:03.025 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rados-debuginfo 2026-02-24T18:30:03.342 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:27 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:03.372 INFO:teuthology.orchestra.run.trial032.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 2.7 MB/s | 1.4 MB 00:00 2026-02-24T18:30:03.373 INFO:teuthology.orchestra.run.trial032.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:03.373 INFO:teuthology.orchestra.run.trial032.stdout:Total 2.7 MB/s | 1.4 MB 00:00 2026-02-24T18:30:03.373 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction check 2026-02-24T18:30:03.375 INFO:teuthology.orchestra.run.trial032.stdout:Transaction check succeeded. 2026-02-24T18:30:03.375 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction test 2026-02-24T18:30:03.397 INFO:teuthology.orchestra.run.trial032.stdout:Transaction test succeeded. 2026-02-24T18:30:03.397 INFO:teuthology.orchestra.run.trial032.stdout:Running transaction 2026-02-24T18:30:03.424 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 779 k 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 2.8 M 2026-02-24T18:30:03.425 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:03.502 INFO:teuthology.orchestra.run.trial032.stdout: Preparing : 1/1 2026-02-24T18:30:03.520 INFO:teuthology.orchestra.run.trial032.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:03.585 INFO:teuthology.orchestra.run.trial032.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout:Installed: 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:30:03.648 INFO:teuthology.orchestra.run.trial032.stdout:Complete! 2026-02-24T18:30:03.676 DEBUG:teuthology.parallel:result is None 2026-02-24T18:30:04.115 INFO:teuthology.orchestra.run.trial112.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 1.1 MB/s | 779 kB 00:00 2026-02-24T18:30:04.115 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:04.115 INFO:teuthology.orchestra.run.trial112.stdout:Total 1.1 MB/s | 779 kB 00:00 2026-02-24T18:30:04.115 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:04.117 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:04.117 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:04.138 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:04.138 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:04.209 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:04.227 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:04.289 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:04.351 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:04.377 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rbd-debuginfo 2026-02-24T18:30:04.694 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:28 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:04.780 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:04.781 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:04.781 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 1.1 M 2026-02-24T18:30:04.781 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 4.3 M 2026-02-24T18:30:04.781 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:05.486 INFO:teuthology.orchestra.run.trial112.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 1.6 MB/s | 1.1 MB 00:00 2026-02-24T18:30:05.487 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:05.487 INFO:teuthology.orchestra.run.trial112.stdout:Total 1.6 MB/s | 1.1 MB 00:00 2026-02-24T18:30:05.487 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:05.489 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:05.489 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:05.511 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:05.511 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:05.601 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:05.619 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:05.680 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:05.743 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:05.769 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install python3-rgw-debuginfo 2026-02-24T18:30:06.088 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:30 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:06.173 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 339 k 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 1.0 M 2026-02-24T18:30:06.174 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:06.724 INFO:teuthology.orchestra.run.trial112.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 617 kB/s | 339 kB 00:00 2026-02-24T18:30:06.724 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:06.724 INFO:teuthology.orchestra.run.trial112.stdout:Total 616 kB/s | 339 kB 00:00 2026-02-24T18:30:06.725 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:06.726 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:06.726 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:06.746 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:06.746 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:06.796 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:06.814 INFO:teuthology.orchestra.run.trial112.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:06.874 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:06.937 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:06.937 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:06.937 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:06.937 INFO:teuthology.orchestra.run.trial112.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:06.938 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:06.938 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:06.964 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-fuse-debuginfo 2026-02-24T18:30:07.278 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:31 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:07.364 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:07.364 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 547 k 2026-02-24T18:30:07.365 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 3.0 M 2026-02-24T18:30:07.366 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:07.970 INFO:teuthology.orchestra.run.trial112.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 904 kB/s | 547 kB 00:00 2026-02-24T18:30:07.970 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:07.971 INFO:teuthology.orchestra.run.trial112.stdout:Total 903 kB/s | 547 kB 00:00 2026-02-24T18:30:07.971 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:07.973 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:07.973 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:07.993 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:07.993 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:08.056 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:08.074 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:08.136 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:08.199 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:08.226 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-mirror-debuginfo 2026-02-24T18:30:08.548 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:32 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:08.633 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:08.634 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:08.634 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:08.634 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:08.634 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:08.634 INFO:teuthology.orchestra.run.trial112.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 54 M 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 353 M 2026-02-24T18:30:08.635 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:28.246 INFO:teuthology.orchestra.run.trial110.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 2.0 MB/s | 298 MB 02:32 2026-02-24T18:30:28.247 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:28.247 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.0 MB/s | 298 MB 02:32 2026-02-24T18:30:28.247 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:30:28.249 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:30:28.249 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:30:28.565 INFO:teuthology.orchestra.run.trial112.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.7 MB/s | 54 MB 00:19 2026-02-24T18:30:28.566 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:28.566 INFO:teuthology.orchestra.run.trial112.stdout:Total 2.7 MB/s | 54 MB 00:19 2026-02-24T18:30:28.566 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:28.568 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:28.568 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:28.744 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:28.745 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:29.144 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:30:29.144 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:30:29.744 INFO:teuthology.orchestra.run.trial018.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 1.3 MB/s | 55 MB 00:42 2026-02-24T18:30:29.745 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:29.745 INFO:teuthology.orchestra.run.trial018.stdout:Total 1.3 MB/s | 55 MB 00:42 2026-02-24T18:30:29.745 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:29.747 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:29.747 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:29.926 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:29.927 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:31.806 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:31.824 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:32.139 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:32.202 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:32.233 DEBUG:teuthology.orchestra.run.trial112:> sudo yum -y install rbd-nbd-debuginfo 2026-02-24T18:30:32.550 INFO:teuthology.orchestra.run.trial112.stdout:Last metadata expiration check: 0:10:56 ago on Tue 24 Feb 2026 06:19:36 PM UTC. 2026-02-24T18:30:32.634 INFO:teuthology.orchestra.run.trial112.stdout:Dependencies resolved. 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout: Package Arch Version Repo Size 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Installing: 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Transaction Summary 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:================================================================================ 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Install 1 Package 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Total download size: 1.4 M 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Installed size: 6.6 M 2026-02-24T18:30:32.635 INFO:teuthology.orchestra.run.trial112.stdout:Downloading Packages: 2026-02-24T18:30:33.066 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:33.085 INFO:teuthology.orchestra.run.trial018.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.401 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.465 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.465 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:33.466 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:33.466 INFO:teuthology.orchestra.run.trial018.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:33.466 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:33.466 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:33.478 INFO:teuthology.orchestra.run.trial112.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 1.6 MB/s | 1.4 MB 00:00 2026-02-24T18:30:33.479 INFO:teuthology.orchestra.run.trial112.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:33.479 INFO:teuthology.orchestra.run.trial112.stdout:Total 1.6 MB/s | 1.4 MB 00:00 2026-02-24T18:30:33.479 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction check 2026-02-24T18:30:33.481 INFO:teuthology.orchestra.run.trial112.stdout:Transaction check succeeded. 2026-02-24T18:30:33.481 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction test 2026-02-24T18:30:33.497 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-cephfs-debuginfo 2026-02-24T18:30:33.504 INFO:teuthology.orchestra.run.trial112.stdout:Transaction test succeeded. 2026-02-24T18:30:33.504 INFO:teuthology.orchestra.run.trial112.stdout:Running transaction 2026-02-24T18:30:33.610 INFO:teuthology.orchestra.run.trial112.stdout: Preparing : 1/1 2026-02-24T18:30:33.628 INFO:teuthology.orchestra.run.trial112.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.692 INFO:teuthology.orchestra.run.trial112.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.755 INFO:teuthology.orchestra.run.trial112.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:33.756 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:33.756 INFO:teuthology.orchestra.run.trial112.stdout:Installed: 2026-02-24T18:30:33.756 INFO:teuthology.orchestra.run.trial112.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:33.756 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:30:33.756 INFO:teuthology.orchestra.run.trial112.stdout:Complete! 2026-02-24T18:30:33.786 DEBUG:teuthology.parallel:result is None 2026-02-24T18:30:33.817 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:10:58 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:33.900 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:33.901 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:33.901 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:33.901 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 628 k 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 2.1 M 2026-02-24T18:30:33.902 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:34.594 INFO:teuthology.orchestra.run.trial018.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 907 kB/s | 628 kB 00:00 2026-02-24T18:30:34.595 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:34.596 INFO:teuthology.orchestra.run.trial018.stdout:Total 905 kB/s | 628 kB 00:00 2026-02-24T18:30:34.596 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:34.597 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:34.597 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:34.617 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:34.617 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:34.682 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:34.700 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:34.760 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:34.823 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:34.824 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:34.824 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:34.824 INFO:teuthology.orchestra.run.trial018.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:34.824 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:34.824 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:34.850 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rados-debuginfo 2026-02-24T18:30:35.166 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:00 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-24T18:30:35.250 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 779 k 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 2.8 M 2026-02-24T18:30:35.251 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:36.074 INFO:teuthology.orchestra.run.trial018.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 945 kB/s | 779 kB 00:00 2026-02-24T18:30:36.075 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:36.075 INFO:teuthology.orchestra.run.trial018.stdout:Total 944 kB/s | 779 kB 00:00 2026-02-24T18:30:36.075 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:36.077 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:36.077 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:36.097 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:36.097 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:36.169 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:36.187 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:36.248 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:36.311 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:30:36.312 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:36.312 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:36.312 INFO:teuthology.orchestra.run.trial018.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:36.312 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:36.312 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:36.338 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rbd-debuginfo 2026-02-24T18:30:36.654 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:01 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:30:36.738 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 1.1 M 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 4.3 M 2026-02-24T18:30:36.739 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:37.368 INFO:teuthology.orchestra.run.trial018.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 1.8 MB/s | 1.1 MB 00:00 2026-02-24T18:30:37.369 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:37.369 INFO:teuthology.orchestra.run.trial018.stdout:Total 1.8 MB/s | 1.1 MB 00:00 2026-02-24T18:30:37.369 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:37.370 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:37.371 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:37.392 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:37.392 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:37.482 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:37.500 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:37.562 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:37.624 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:37.624 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:37.625 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:37.625 INFO:teuthology.orchestra.run.trial018.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:37.625 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:37.625 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:37.652 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install python3-rgw-debuginfo 2026-02-24T18:30:37.971 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:02 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:38.055 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:38.056 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:38.056 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:38.056 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:38.056 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:38.056 INFO:teuthology.orchestra.run.trial018.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 339 k 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 1.0 M 2026-02-24T18:30:38.057 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:38.791 INFO:teuthology.orchestra.run.trial018.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 462 kB/s | 339 kB 00:00 2026-02-24T18:30:38.791 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:38.791 INFO:teuthology.orchestra.run.trial018.stdout:Total 461 kB/s | 339 kB 00:00 2026-02-24T18:30:38.791 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:38.793 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:38.793 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:38.812 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:38.812 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:38.862 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:38.880 INFO:teuthology.orchestra.run.trial018.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:38.940 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:39.003 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:30:39.004 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:39.004 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:39.004 INFO:teuthology.orchestra.run.trial018.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:39.004 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:39.004 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:39.031 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-fuse-debuginfo 2026-02-24T18:30:39.350 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:04 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-24T18:30:39.435 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 547 k 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 3.0 M 2026-02-24T18:30:39.436 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:40.057 INFO:teuthology.orchestra.run.trial018.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 881 kB/s | 547 kB 00:00 2026-02-24T18:30:40.057 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:40.057 INFO:teuthology.orchestra.run.trial018.stdout:Total 880 kB/s | 547 kB 00:00 2026-02-24T18:30:40.057 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:40.059 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:40.059 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:40.079 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:40.079 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:40.144 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:40.162 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:40.224 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:40.287 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:40.287 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:40.288 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:40.288 INFO:teuthology.orchestra.run.trial018.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:40.288 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:40.288 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:40.315 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-mirror-debuginfo 2026-02-24T18:30:40.632 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:05 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:40.716 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:40.717 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:40.717 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:40.717 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:40.717 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:40.718 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:40.719 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 54 M 2026-02-24T18:30:40.719 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 353 M 2026-02-24T18:30:40.719 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:44.145 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:30:44.166 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:44.880 INFO:teuthology.orchestra.run.trial018.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 13 MB/s | 54 MB 00:04 2026-02-24T18:30:44.880 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:44.880 INFO:teuthology.orchestra.run.trial018.stdout:Total 13 MB/s | 54 MB 00:04 2026-02-24T18:30:44.880 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:44.882 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:44.882 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:45.059 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:45.059 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:45.194 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:45.257 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:45.258 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:45.258 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:30:45.258 INFO:teuthology.orchestra.run.trial110.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:45.258 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:45.258 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:30:45.309 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install ceph-fuse-debuginfo 2026-02-24T18:30:45.626 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:08 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:30:45.709 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:30:45.709 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:45.710 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 6.8 M 2026-02-24T18:30:45.711 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 36 M 2026-02-24T18:30:45.711 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:30:48.156 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:48.174 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:48.488 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:48.551 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:48.551 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:48.551 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:48.551 INFO:teuthology.orchestra.run.trial018.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:48.552 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:48.552 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:48.583 DEBUG:teuthology.orchestra.run.trial018:> sudo yum -y install rbd-nbd-debuginfo 2026-02-24T18:30:48.599 INFO:teuthology.orchestra.run.trial110.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 2.4 MB/s | 6.8 MB 00:02 2026-02-24T18:30:48.600 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:48.600 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.4 MB/s | 6.8 MB 00:02 2026-02-24T18:30:48.600 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:30:48.601 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:30:48.602 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:30:48.639 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:30:48.640 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:30:48.903 INFO:teuthology.orchestra.run.trial018.stdout:Last metadata expiration check: 0:11:13 ago on Tue 24 Feb 2026 06:19:35 PM UTC. 2026-02-24T18:30:48.987 INFO:teuthology.orchestra.run.trial018.stdout:Dependencies resolved. 2026-02-24T18:30:48.987 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout: Package Arch Version Repo Size 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:Installing: 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:Transaction Summary 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:================================================================================ 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:Install 1 Package 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:Total download size: 1.4 M 2026-02-24T18:30:48.988 INFO:teuthology.orchestra.run.trial018.stdout:Installed size: 6.6 M 2026-02-24T18:30:48.989 INFO:teuthology.orchestra.run.trial018.stdout:Downloading Packages: 2026-02-24T18:30:49.052 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:30:49.070 INFO:teuthology.orchestra.run.trial110.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:30:49.165 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:49.228 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:30:49.255 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librados-devel-debuginfo 2026-02-24T18:30:49.576 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:12 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:30:49.660 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:49.661 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 112 k 2026-02-24T18:30:49.662 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 493 k 2026-02-24T18:30:49.662 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:30:50.105 INFO:teuthology.orchestra.run.trial110.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 253 kB/s | 112 kB 00:00 2026-02-24T18:30:50.105 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:50.105 INFO:teuthology.orchestra.run.trial110.stdout:Total 252 kB/s | 112 kB 00:00 2026-02-24T18:30:50.105 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:30:50.107 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:30:50.107 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:30:50.125 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:30:50.125 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:30:50.165 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:30:50.183 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:50.241 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:50.282 INFO:teuthology.orchestra.run.trial018.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 1.0 MB/s | 1.4 MB 00:01 2026-02-24T18:30:50.283 INFO:teuthology.orchestra.run.trial018.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:50.283 INFO:teuthology.orchestra.run.trial018.stdout:Total 1.0 MB/s | 1.4 MB 00:01 2026-02-24T18:30:50.283 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction check 2026-02-24T18:30:50.285 INFO:teuthology.orchestra.run.trial018.stdout:Transaction check succeeded. 2026-02-24T18:30:50.285 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction test 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:50.304 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:30:50.307 INFO:teuthology.orchestra.run.trial018.stdout:Transaction test succeeded. 2026-02-24T18:30:50.307 INFO:teuthology.orchestra.run.trial018.stdout:Running transaction 2026-02-24T18:30:50.333 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install libcephfs2-debuginfo 2026-02-24T18:30:50.412 INFO:teuthology.orchestra.run.trial018.stdout: Preparing : 1/1 2026-02-24T18:30:50.430 INFO:teuthology.orchestra.run.trial018.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:50.493 INFO:teuthology.orchestra.run.trial018.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout:Installed: 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:30:50.557 INFO:teuthology.orchestra.run.trial018.stdout:Complete! 2026-02-24T18:30:50.592 DEBUG:teuthology.parallel:result is None 2026-02-24T18:30:50.651 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:13 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:30:50.734 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:30:50.735 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 6.0 M 2026-02-24T18:30:50.736 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 31 M 2026-02-24T18:30:50.737 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:30:53.133 INFO:teuthology.orchestra.run.trial110.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.5 MB/s | 6.0 MB 00:02 2026-02-24T18:30:53.134 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:53.134 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.5 MB/s | 6.0 MB 00:02 2026-02-24T18:30:53.134 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:30:53.136 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:30:53.136 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:30:53.171 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:30:53.171 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:30:53.529 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:30:53.548 INFO:teuthology.orchestra.run.trial110.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:53.628 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:53.690 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:30:53.691 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:53.691 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:30:53.691 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:30:53.691 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:53.691 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:30:53.718 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librados2-debuginfo 2026-02-24T18:30:54.036 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:17 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:30:54.120 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 47 M 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 302 M 2026-02-24T18:30:54.121 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:30:57.819 INFO:teuthology.orchestra.run.trial110.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 13 MB/s | 47 MB 00:03 2026-02-24T18:30:57.820 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:30:57.820 INFO:teuthology.orchestra.run.trial110.stdout:Total 13 MB/s | 47 MB 00:03 2026-02-24T18:30:57.820 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:30:57.822 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:30:57.822 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:30:57.981 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:30:57.982 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:00.677 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:00.695 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:31:01.003 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:01.066 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:01.097 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install librbd1-debuginfo 2026-02-24T18:31:01.410 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:24 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:01.493 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:01.494 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 55 M 2026-02-24T18:31:01.495 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 374 M 2026-02-24T18:31:01.495 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:17.261 INFO:teuthology.orchestra.run.trial110.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 3.5 MB/s | 55 MB 00:15 2026-02-24T18:31:17.262 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:17.263 INFO:teuthology.orchestra.run.trial110.stdout:Total 3.5 MB/s | 55 MB 00:15 2026-02-24T18:31:17.263 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:17.264 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:17.265 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:17.447 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:17.447 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:20.591 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:20.611 INFO:teuthology.orchestra.run.trial110.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:20.941 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:21.005 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:21.040 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-cephfs-debuginfo 2026-02-24T18:31:21.365 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:44 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:21.448 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:21.449 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:21.450 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:21.450 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:21.450 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 628 k 2026-02-24T18:31:21.450 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 2.1 M 2026-02-24T18:31:21.450 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:22.780 INFO:teuthology.orchestra.run.trial110.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 472 kB/s | 628 kB 00:01 2026-02-24T18:31:22.781 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:22.781 INFO:teuthology.orchestra.run.trial110.stdout:Total 472 kB/s | 628 kB 00:01 2026-02-24T18:31:22.781 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:22.783 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:22.783 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:22.804 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:22.804 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:22.870 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:22.888 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:31:22.949 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:23.012 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:23.038 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rados-debuginfo 2026-02-24T18:31:23.356 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:46 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:23.439 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:23.440 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 779 k 2026-02-24T18:31:23.440 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 2.8 M 2026-02-24T18:31:23.440 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:24.211 INFO:teuthology.orchestra.run.trial110.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 1.0 MB/s | 779 kB 00:00 2026-02-24T18:31:24.213 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:24.213 INFO:teuthology.orchestra.run.trial110.stdout:Total 1.0 MB/s | 779 kB 00:00 2026-02-24T18:31:24.213 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:24.215 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:24.215 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:24.236 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:24.236 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:24.309 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:24.327 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:31:24.388 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:24.452 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:24.479 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rbd-debuginfo 2026-02-24T18:31:24.794 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:47 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:24.877 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:24.878 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:24.879 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 1.1 M 2026-02-24T18:31:24.879 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 4.3 M 2026-02-24T18:31:24.879 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:27.868 INFO:teuthology.orchestra.run.trial110.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 390 kB/s | 1.1 MB 00:02 2026-02-24T18:31:27.868 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:27.868 INFO:teuthology.orchestra.run.trial110.stdout:Total 390 kB/s | 1.1 MB 00:02 2026-02-24T18:31:27.868 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:27.870 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:27.870 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:27.892 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:27.892 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:27.982 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:28.000 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:28.066 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:28.129 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:28.156 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install python3-rgw-debuginfo 2026-02-24T18:31:28.475 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:51 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:28.561 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:28.561 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:28.561 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:28.561 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:28.561 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 339 k 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 1.0 M 2026-02-24T18:31:28.562 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:29.033 INFO:teuthology.orchestra.run.trial110.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 718 kB/s | 339 kB 00:00 2026-02-24T18:31:29.034 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:29.034 INFO:teuthology.orchestra.run.trial110.stdout:Total 716 kB/s | 339 kB 00:00 2026-02-24T18:31:29.034 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:29.036 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:29.036 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:29.055 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:29.055 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:29.105 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:29.123 INFO:teuthology.orchestra.run.trial110.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:29.185 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:29.248 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:29.276 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-fuse-debuginfo 2026-02-24T18:31:29.600 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:52 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:29.684 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:29.684 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:29.684 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:29.684 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 547 k 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 3.0 M 2026-02-24T18:31:29.685 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:30.264 INFO:teuthology.orchestra.run.trial110.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 945 kB/s | 547 kB 00:00 2026-02-24T18:31:30.264 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:30.264 INFO:teuthology.orchestra.run.trial110.stdout:Total 944 kB/s | 547 kB 00:00 2026-02-24T18:31:30.264 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:30.266 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:30.266 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:30.286 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:30.286 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:30.351 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:30.368 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:30.431 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:30.494 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:30.495 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:30.495 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:30.495 INFO:teuthology.orchestra.run.trial110.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:30.495 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:30.495 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:30.522 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-mirror-debuginfo 2026-02-24T18:31:30.846 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:11:53 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 54 M 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 353 M 2026-02-24T18:31:30.931 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:49.532 INFO:teuthology.orchestra.run.trial110.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.9 MB/s | 54 MB 00:18 2026-02-24T18:31:49.532 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:49.533 INFO:teuthology.orchestra.run.trial110.stdout:Total 2.9 MB/s | 54 MB 00:18 2026-02-24T18:31:49.533 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:49.535 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:49.535 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:49.711 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:49.711 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:52.757 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:52.775 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:31:53.081 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:53.144 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:53.176 DEBUG:teuthology.orchestra.run.trial110:> sudo yum -y install rbd-nbd-debuginfo 2026-02-24T18:31:53.492 INFO:teuthology.orchestra.run.trial110.stdout:Last metadata expiration check: 0:12:16 ago on Tue 24 Feb 2026 06:19:37 PM UTC. 2026-02-24T18:31:53.575 INFO:teuthology.orchestra.run.trial110.stdout:Dependencies resolved. 2026-02-24T18:31:53.575 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout: Package Arch Version Repo Size 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:Installing: 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:Transaction Summary 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:================================================================================ 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:Install 1 Package 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:53.576 INFO:teuthology.orchestra.run.trial110.stdout:Total download size: 1.4 M 2026-02-24T18:31:53.577 INFO:teuthology.orchestra.run.trial110.stdout:Installed size: 6.6 M 2026-02-24T18:31:53.577 INFO:teuthology.orchestra.run.trial110.stdout:Downloading Packages: 2026-02-24T18:31:54.492 INFO:teuthology.orchestra.run.trial110.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 1.5 MB/s | 1.4 MB 00:00 2026-02-24T18:31:54.493 INFO:teuthology.orchestra.run.trial110.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:31:54.493 INFO:teuthology.orchestra.run.trial110.stdout:Total 1.5 MB/s | 1.4 MB 00:00 2026-02-24T18:31:54.493 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction check 2026-02-24T18:31:54.495 INFO:teuthology.orchestra.run.trial110.stdout:Transaction check succeeded. 2026-02-24T18:31:54.495 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction test 2026-02-24T18:31:54.516 INFO:teuthology.orchestra.run.trial110.stdout:Transaction test succeeded. 2026-02-24T18:31:54.516 INFO:teuthology.orchestra.run.trial110.stdout:Running transaction 2026-02-24T18:31:54.622 INFO:teuthology.orchestra.run.trial110.stdout: Preparing : 1/1 2026-02-24T18:31:54.639 INFO:teuthology.orchestra.run.trial110.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:54.706 INFO:teuthology.orchestra.run.trial110.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:54.769 INFO:teuthology.orchestra.run.trial110.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:31:54.769 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:54.769 INFO:teuthology.orchestra.run.trial110.stdout:Installed: 2026-02-24T18:31:54.770 INFO:teuthology.orchestra.run.trial110.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:31:54.770 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:31:54.770 INFO:teuthology.orchestra.run.trial110.stdout:Complete! 2026-02-24T18:31:54.802 DEBUG:teuthology.parallel:result is None 2026-02-24T18:31:54.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:31:54.996 DEBUG:teuthology.orchestra.run.trial018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T18:31:55.028 INFO:teuthology.orchestra.run.trial018.stdout:19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.028 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.028 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-24T18:31:55.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:31:55.160 DEBUG:teuthology.orchestra.run.trial032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T18:31:55.192 INFO:teuthology.orchestra.run.trial032.stdout:19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.193 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.193 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-24T18:31:55.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:31:55.365 DEBUG:teuthology.orchestra.run.trial110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T18:31:55.386 INFO:teuthology.orchestra.run.trial110.stdout:19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.386 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.386 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-24T18:31:55.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:31:55.510 DEBUG:teuthology.orchestra.run.trial112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T18:31:55.541 INFO:teuthology.orchestra.run.trial112.stdout:19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.542 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-24T18:31:55.542 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-24T18:31:55.543 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T18:31:55.543 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:31:55.543 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:31:55.570 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:31:55.571 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:31:55.598 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:31:55.598 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:31:55.625 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:31:55.625 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:31:55.652 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T18:31:55.653 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:31:55.653 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:31:55.678 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:31:55.742 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:31:55.742 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:31:55.767 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:31:55.832 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:31:55.832 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:31:55.858 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:31:55.923 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:31:55.924 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:31:55.949 DEBUG:teuthology.orchestra.run.trial112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:31:56.015 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T18:31:56.015 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:31:56.015 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:31:56.040 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:31:56.104 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:31:56.104 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:31:56.129 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:31:56.194 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:31:56.194 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:31:56.220 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:31:56.283 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:31:56.283 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:31:56.308 DEBUG:teuthology.orchestra.run.trial112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:31:56.373 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T18:31:56.374 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:31:56.374 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:31:56.400 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:31:56.464 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:31:56.464 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:31:56.490 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:31:56.554 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:31:56.555 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:31:56.580 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:31:56.645 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:31:56.646 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:31:56.671 DEBUG:teuthology.orchestra.run.trial112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:31:56.735 INFO:teuthology.run_tasks:Running task ceph... 2026-02-24T18:31:56.846 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-24T18:31:56.846 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod 777 /var/log/ceph 2026-02-24T18:31:56.848 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod 777 /var/log/ceph 2026-02-24T18:31:56.850 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 777 /var/log/ceph 2026-02-24T18:31:56.851 DEBUG:teuthology.orchestra.run.trial112:> sudo chmod 777 /var/log/ceph 2026-02-24T18:31:56.875 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-24T18:31:56.875 DEBUG:teuthology.orchestra.run.trial018:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T18:31:56.913 DEBUG:teuthology.orchestra.run.trial032:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T18:31:56.915 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T18:31:56.916 DEBUG:teuthology.orchestra.run.trial112:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T18:31:56.940 INFO:tasks.ceph:Creating extra log directories... 2026-02-24T18:31:56.940 DEBUG:teuthology.orchestra.run.trial018:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T18:31:56.976 DEBUG:teuthology.orchestra.run.trial032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T18:31:56.978 DEBUG:teuthology.orchestra.run.trial110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T18:31:56.981 DEBUG:teuthology.orchestra.run.trial112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T18:31:57.024 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-24T18:31:57.024 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-24T18:31:57.024 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '68651', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'stop-daemons-timeout': 1200, 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 5907, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial018.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiOv4wxAd5q5Cfw+X+/1OArumqTctOp8Jn+J/erK9HsZligMjUkVxX0bN61dka3O7FWelZ7pyRxhDWc1V3KNsQ=', 'trial032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL8xNQnp/KmEH/gR6flU2UiYBKNnUKXkFMcfIQvOqLhf10RfYLsG2h5Hn3sZHPqT1IT849eIPOew1mc5hPHjRwc=', 'trial110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCaNdVXemvq4+d7G9yuKi7aoK94ARoZOpiql99rbT7YanP/B/wKy6n0UffTwiZRa5B8E22IZTPPYQE6yUZa9dXI=', 'trial112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/B8xTpilT/W17zf15Y8jwxEsmW/8o+yZwlILDFqq3pw7dT+fodE953+WOhU6hPpeQtL5kw5gLOeY0p40lQ4Qw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["-- Don't run this test on tiny clusters\nif py_len(yaml.roles) < 4 then reject() end\n"]}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-24_16:22:43', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-24T18:31:57.025 DEBUG:teuthology.orchestra.run.trial018:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T18:31:57.040 DEBUG:teuthology.orchestra.run.trial032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T18:31:57.042 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T18:31:57.044 DEBUG:teuthology.orchestra.run.trial112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T18:31:57.062 DEBUG:teuthology.orchestra.run.trial018:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T18:31:57.096 DEBUG:teuthology.orchestra.run.trial032:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T18:31:57.098 DEBUG:teuthology.orchestra.run.trial110:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T18:31:57.100 DEBUG:teuthology.orchestra.run.trial112:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T18:31:57.125 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:31:57.125 DEBUG:teuthology.orchestra.run.trial018:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T18:31:57.175 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T18:31:57.175 DEBUG:teuthology.orchestra.run.trial018:> stat /dev/vg_nvme/lv_1 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:57.231 INFO:teuthology.orchestra.run.trial018.stdout:Access: 2026-02-24 18:20:30.309576748 +0000 2026-02-24T18:31:57.232 INFO:teuthology.orchestra.run.trial018.stdout:Modify: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.232 INFO:teuthology.orchestra.run.trial018.stdout:Change: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.232 INFO:teuthology.orchestra.run.trial018.stdout: Birth: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.232 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T18:31:57.293 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records in 2026-02-24T18:31:57.294 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records out 2026-02-24T18:31:57.294 INFO:teuthology.orchestra.run.trial018.stderr:512 bytes copied, 0.000149878 s, 3.4 MB/s 2026-02-24T18:31:57.295 DEBUG:teuthology.orchestra.run.trial018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T18:31:57.351 DEBUG:teuthology.orchestra.run.trial018:> stat /dev/vg_nvme/lv_2 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Access: 2026-02-24 18:20:30.310576710 +0000 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Modify: 2026-02-24 18:19:44.635357827 +0000 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout:Change: 2026-02-24 18:19:44.635357827 +0000 2026-02-24T18:31:57.407 INFO:teuthology.orchestra.run.trial018.stdout: Birth: 2026-02-24 18:19:44.635357827 +0000 2026-02-24T18:31:57.407 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T18:31:57.469 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records in 2026-02-24T18:31:57.469 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records out 2026-02-24T18:31:57.469 INFO:teuthology.orchestra.run.trial018.stderr:512 bytes copied, 0.000142735 s, 3.6 MB/s 2026-02-24T18:31:57.471 DEBUG:teuthology.orchestra.run.trial018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T18:31:57.528 DEBUG:teuthology.orchestra.run.trial018:> stat /dev/vg_nvme/lv_3 2026-02-24T18:31:57.585 INFO:teuthology.orchestra.run.trial018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T18:31:57.585 INFO:teuthology.orchestra.run.trial018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:57.585 INFO:teuthology.orchestra.run.trial018.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout:Access: 2026-02-24 18:20:30.310576710 +0000 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout:Modify: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout:Change: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.586 INFO:teuthology.orchestra.run.trial018.stdout: Birth: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.586 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T18:31:57.648 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records in 2026-02-24T18:31:57.648 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records out 2026-02-24T18:31:57.648 INFO:teuthology.orchestra.run.trial018.stderr:512 bytes copied, 0.000145089 s, 3.5 MB/s 2026-02-24T18:31:57.649 DEBUG:teuthology.orchestra.run.trial018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T18:31:57.706 DEBUG:teuthology.orchestra.run.trial018:> stat /dev/vg_nvme/lv_4 2026-02-24T18:31:57.762 INFO:teuthology.orchestra.run.trial018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T18:31:57.762 INFO:teuthology.orchestra.run.trial018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:57.762 INFO:teuthology.orchestra.run.trial018.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout:Access: 2026-02-24 18:20:30.310576710 +0000 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout:Modify: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout:Change: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.763 INFO:teuthology.orchestra.run.trial018.stdout: Birth: 2026-02-24 18:19:44.636357788 +0000 2026-02-24T18:31:57.763 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T18:31:57.825 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records in 2026-02-24T18:31:57.825 INFO:teuthology.orchestra.run.trial018.stderr:1+0 records out 2026-02-24T18:31:57.825 INFO:teuthology.orchestra.run.trial018.stderr:512 bytes copied, 0.000136293 s, 3.8 MB/s 2026-02-24T18:31:57.827 DEBUG:teuthology.orchestra.run.trial018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T18:31:57.883 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-24T18:31:57.883 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:31:57.883 DEBUG:teuthology.orchestra.run.trial032:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T18:31:57.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T18:31:57.900 DEBUG:teuthology.orchestra.run.trial032:> stat /dev/vg_nvme/lv_1 2026-02-24T18:31:57.956 INFO:teuthology.orchestra.run.trial032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Access: 2026-02-24 18:20:35.346845257 +0000 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Modify: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout:Change: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:57.957 INFO:teuthology.orchestra.run.trial032.stdout: Birth: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:57.958 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T18:31:58.019 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records in 2026-02-24T18:31:58.019 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records out 2026-02-24T18:31:58.019 INFO:teuthology.orchestra.run.trial032.stderr:512 bytes copied, 0.000162542 s, 3.1 MB/s 2026-02-24T18:31:58.021 DEBUG:teuthology.orchestra.run.trial032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T18:31:58.078 DEBUG:teuthology.orchestra.run.trial032:> stat /dev/vg_nvme/lv_2 2026-02-24T18:31:58.135 INFO:teuthology.orchestra.run.trial032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T18:31:58.135 INFO:teuthology.orchestra.run.trial032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:58.135 INFO:teuthology.orchestra.run.trial032.stdout:Device: 6h/6d Inode: 1296 Links: 1 2026-02-24T18:31:58.135 INFO:teuthology.orchestra.run.trial032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:58.135 INFO:teuthology.orchestra.run.trial032.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:58.136 INFO:teuthology.orchestra.run.trial032.stdout:Access: 2026-02-24 18:20:35.346845257 +0000 2026-02-24T18:31:58.136 INFO:teuthology.orchestra.run.trial032.stdout:Modify: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.136 INFO:teuthology.orchestra.run.trial032.stdout:Change: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.136 INFO:teuthology.orchestra.run.trial032.stdout: Birth: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.136 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T18:31:58.198 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records in 2026-02-24T18:31:58.199 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records out 2026-02-24T18:31:58.199 INFO:teuthology.orchestra.run.trial032.stderr:512 bytes copied, 0.000143556 s, 3.6 MB/s 2026-02-24T18:31:58.200 DEBUG:teuthology.orchestra.run.trial032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T18:31:58.257 DEBUG:teuthology.orchestra.run.trial032:> stat /dev/vg_nvme/lv_3 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout:Access: 2026-02-24 18:20:35.347845245 +0000 2026-02-24T18:31:58.314 INFO:teuthology.orchestra.run.trial032.stdout:Modify: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.315 INFO:teuthology.orchestra.run.trial032.stdout:Change: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.315 INFO:teuthology.orchestra.run.trial032.stdout: Birth: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.315 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T18:31:58.376 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records in 2026-02-24T18:31:58.377 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records out 2026-02-24T18:31:58.377 INFO:teuthology.orchestra.run.trial032.stderr:512 bytes copied, 0.000141122 s, 3.6 MB/s 2026-02-24T18:31:58.378 DEBUG:teuthology.orchestra.run.trial032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T18:31:58.436 DEBUG:teuthology.orchestra.run.trial032:> stat /dev/vg_nvme/lv_4 2026-02-24T18:31:58.492 INFO:teuthology.orchestra.run.trial032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T18:31:58.492 INFO:teuthology.orchestra.run.trial032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Access: 2026-02-24 18:20:35.347845245 +0000 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Modify: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout:Change: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.493 INFO:teuthology.orchestra.run.trial032.stdout: Birth: 2026-02-24 18:19:51.508378389 +0000 2026-02-24T18:31:58.494 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T18:31:58.555 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records in 2026-02-24T18:31:58.556 INFO:teuthology.orchestra.run.trial032.stderr:1+0 records out 2026-02-24T18:31:58.556 INFO:teuthology.orchestra.run.trial032.stderr:512 bytes copied, 0.0001399 s, 3.7 MB/s 2026-02-24T18:31:58.557 DEBUG:teuthology.orchestra.run.trial032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T18:31:58.615 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-24T18:31:58.616 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:31:58.616 DEBUG:teuthology.orchestra.run.trial110:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T18:31:58.635 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T18:31:58.635 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_1 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:58.691 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 18:20:36.789603890 +0000 2026-02-24T18:31:58.692 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.692 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.692 INFO:teuthology.orchestra.run.trial110.stdout: Birth: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.692 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T18:31:58.754 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T18:31:58.754 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T18:31:58.754 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000157503 s, 3.3 MB/s 2026-02-24T18:31:58.755 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T18:31:58.813 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_2 2026-02-24T18:31:58.869 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 18:20:36.790603893 +0000 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.870 INFO:teuthology.orchestra.run.trial110.stdout: Birth: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:58.871 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T18:31:58.932 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T18:31:58.933 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T18:31:58.933 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000147254 s, 3.5 MB/s 2026-02-24T18:31:58.935 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T18:31:58.992 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_3 2026-02-24T18:31:59.049 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 18:20:36.790603893 +0000 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:59.050 INFO:teuthology.orchestra.run.trial110.stdout: Birth: 2026-02-24 18:19:49.124086981 +0000 2026-02-24T18:31:59.051 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T18:31:59.111 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T18:31:59.112 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T18:31:59.112 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000150038 s, 3.4 MB/s 2026-02-24T18:31:59.113 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T18:31:59.170 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_4 2026-02-24T18:31:59.224 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 18:20:36.790603893 +0000 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 18:19:49.125086944 +0000 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 18:19:49.125086944 +0000 2026-02-24T18:31:59.225 INFO:teuthology.orchestra.run.trial110.stdout: Birth: 2026-02-24 18:19:49.125086944 +0000 2026-02-24T18:31:59.225 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T18:31:59.287 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T18:31:59.287 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T18:31:59.287 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000169655 s, 3.0 MB/s 2026-02-24T18:31:59.288 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T18:31:59.345 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-24T18:31:59.345 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:31:59.345 DEBUG:teuthology.orchestra.run.trial112:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T18:31:59.363 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T18:31:59.363 DEBUG:teuthology.orchestra.run.trial112:> stat /dev/vg_nvme/lv_1 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.421 INFO:teuthology.orchestra.run.trial112.stdout:Access: 2026-02-24 18:20:34.587153292 +0000 2026-02-24T18:31:59.422 INFO:teuthology.orchestra.run.trial112.stdout:Modify: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.422 INFO:teuthology.orchestra.run.trial112.stdout:Change: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.422 INFO:teuthology.orchestra.run.trial112.stdout: Birth: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.422 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T18:31:59.484 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records in 2026-02-24T18:31:59.484 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records out 2026-02-24T18:31:59.484 INFO:teuthology.orchestra.run.trial112.stderr:512 bytes copied, 0.000239096 s, 2.1 MB/s 2026-02-24T18:31:59.486 DEBUG:teuthology.orchestra.run.trial112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T18:31:59.543 DEBUG:teuthology.orchestra.run.trial112:> stat /dev/vg_nvme/lv_2 2026-02-24T18:31:59.601 INFO:teuthology.orchestra.run.trial112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T18:31:59.601 INFO:teuthology.orchestra.run.trial112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.601 INFO:teuthology.orchestra.run.trial112.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-24T18:31:59.601 INFO:teuthology.orchestra.run.trial112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.601 INFO:teuthology.orchestra.run.trial112.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.602 INFO:teuthology.orchestra.run.trial112.stdout:Access: 2026-02-24 18:20:34.587153292 +0000 2026-02-24T18:31:59.602 INFO:teuthology.orchestra.run.trial112.stdout:Modify: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.602 INFO:teuthology.orchestra.run.trial112.stdout:Change: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.602 INFO:teuthology.orchestra.run.trial112.stdout: Birth: 2026-02-24 18:19:50.643802809 +0000 2026-02-24T18:31:59.602 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T18:31:59.663 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records in 2026-02-24T18:31:59.663 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records out 2026-02-24T18:31:59.663 INFO:teuthology.orchestra.run.trial112.stderr:512 bytes copied, 0.000213448 s, 2.4 MB/s 2026-02-24T18:31:59.664 DEBUG:teuthology.orchestra.run.trial112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T18:31:59.722 DEBUG:teuthology.orchestra.run.trial112:> stat /dev/vg_nvme/lv_3 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Access: 2026-02-24 18:20:34.587153292 +0000 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Modify: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout:Change: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.778 INFO:teuthology.orchestra.run.trial112.stdout: Birth: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.778 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T18:31:59.842 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records in 2026-02-24T18:31:59.842 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records out 2026-02-24T18:31:59.842 INFO:teuthology.orchestra.run.trial112.stderr:512 bytes copied, 0.000157534 s, 3.3 MB/s 2026-02-24T18:31:59.843 DEBUG:teuthology.orchestra.run.trial112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T18:31:59.900 DEBUG:teuthology.orchestra.run.trial112:> stat /dev/vg_nvme/lv_4 2026-02-24T18:31:59.956 INFO:teuthology.orchestra.run.trial112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Device: 6h/6d Inode: 1287 Links: 1 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Access: 2026-02-24 18:20:34.587153292 +0000 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Modify: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout:Change: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.957 INFO:teuthology.orchestra.run.trial112.stdout: Birth: 2026-02-24 18:19:50.641802886 +0000 2026-02-24T18:31:59.958 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T18:32:00.019 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records in 2026-02-24T18:32:00.020 INFO:teuthology.orchestra.run.trial112.stderr:1+0 records out 2026-02-24T18:32:00.020 INFO:teuthology.orchestra.run.trial112.stderr:512 bytes copied, 0.00015017 s, 3.4 MB/s 2026-02-24T18:32:00.021 DEBUG:teuthology.orchestra.run.trial112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T18:32:00.079 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:00.079 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial018.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial032.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial110.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial112.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-24T18:32:00.079 INFO:tasks.ceph:Generating config... 2026-02-24T18:32:00.082 INFO:tasks.ceph:[client] debug monc = 20 2026-02-24T18:32:00.082 INFO:tasks.ceph:[client] debug ms = 1 2026-02-24T18:32:00.082 INFO:tasks.ceph:[client] debug objecter = 20 2026-02-24T18:32:00.082 INFO:tasks.ceph:[client] debug rados = 20 2026-02-24T18:32:00.082 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-24T18:32:00.082 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-24T18:32:00.082 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-24T18:32:00.082 INFO:tasks.ceph:[global] ms type = async 2026-02-24T18:32:00.082 INFO:tasks.ceph:[global] osd heartbeat grace = 160 2026-02-24T18:32:00.083 INFO:tasks.ceph:[global] osd op thread timeout = 150 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-24T18:32:00.083 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-24T18:32:00.084 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2026-02-24T18:32:00.084 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] debug bluefs = 1 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] debug bluestore = 1 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-24T18:32:00.084 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd class default list = * 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd class load list = * 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd client watch timeout = 240 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd fast shutdown = False 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-24T18:32:00.085 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-24T18:32:00.086 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-24T18:32:00.086 INFO:tasks.ceph:Setting up mon.a... 2026-02-24T18:32:00.086 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-24T18:32:00.122 INFO:teuthology.orchestra.run.trial018.stdout:creating /etc/ceph/ceph.keyring 2026-02-24T18:32:00.125 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-24T18:32:00.200 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:32:00.263 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.18'), ('mon.b', '10.20.193.32'), ('mon.c', '10.20.193.110')] 2026-02-24T18:32:00.264 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.18,10.20.193.32,10.20.193.110', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-24T18:32:00.266 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:32:00.266 DEBUG:teuthology.orchestra.run.trial018:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T18:32:00.320 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.18 --add b 10.20.193.32 --add c 10.20.193.110 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:monmaptool: generated fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:setting min_mon_release = quincy 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:epoch 0 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:32:00.394 INFO:teuthology.orchestra.run.trial018.stdout:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:min_mon_release 17 (quincy) 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:election_strategy: 1 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:32:00.395 INFO:teuthology.orchestra.run.trial018.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-24T18:32:00.397 DEBUG:teuthology.orchestra.run.trial018:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T18:32:00.453 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1... 2026-02-24T18:32:00.454 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T18:32:00.494 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T18:32:00.496 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T18:32:00.498 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout:[global] 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout: chdir = "" 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout: auth supported = cephx 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.530 INFO:teuthology.orchestra.run.trial018.stdout: filestore xattr use omap = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon clock drift allowed = 1.000 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: osd crush chooseleaf type = 0 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: auth debug = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: ms die on old message = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: ms die on bug = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon pg warn max object skew = 0 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: osd pool default size = 2 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon osd allow primary affinity = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon osd allow pg remap = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on legacy crush tunables = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on crush straw calc version zero = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on no sortbitwise = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on osd down out interval zero = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on too few osds = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon_allow_pool_size_one = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: osd default data pool replay window = 5 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon allow pool delete = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon cluster log file level = debug 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: debug asserts on shutdown = true 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon health detail to clog = false 2026-02-24T18:32:00.531 INFO:teuthology.orchestra.run.trial018.stdout: mon host = "10.20.193.18,10.20.193.32,10.20.193.110" 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: mon client directed command retry = 5 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: mon election default strategy = 1 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: ms inject socket failures = 5000 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: ms type = async 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd heartbeat grace = 160 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd op thread timeout = 150 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: fsid = 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout:[osd] 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd journal size = 100 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd scrub load threshold = 5.0 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd scrub max interval = 600 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd mclock profile = high_recovery_ops 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd mclock skip benchmark = True 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd recover clone overlap = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd recovery max chunk = 1048576 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug shutdown = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug op order = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug verify stray on activate = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug trim objects = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd open classes on start = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug pg log writeout = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd deep scrub update digest min age = 30 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd map max advance = 10 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: journal zero on create = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: filestore ondisk finisher threads = 3 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: filestore apply finisher threads = 3 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: bdev debug aio = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: osd debug misdirected ops = true 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: bdev async discard = True 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: bdev enable discard = True 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: bluestore block size = 96636764160 2026-02-24T18:32:00.532 INFO:teuthology.orchestra.run.trial018.stdout: bluestore fsck on mount = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: bluestore zero block detection = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug bluefs = 1 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug bluestore = 1 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug monc = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug ms = 1 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug osd = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug rocksdb = 10 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon osd full ratio = 0.9 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon osd nearfull ratio = 0.8 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd class default list = * 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd class load list = * 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd client watch timeout = 240 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd debug verify cached snaps = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd debug verify missing on start = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd failsafe full ratio = 0.95 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd fast shutdown = False 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd mclock override recovery settings = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd memory target = 1610612736 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd objectstore = bluestore 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd op queue = debug_random 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd op queue cut off = debug_random 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: osd shutdown pgref assert = True 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout:[mgr] 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug ms = 1 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug mgr = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug mon = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug auth = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mgr/telemetry/nag = false 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout:[mon] 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug ms = 1 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug mon = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug paxos = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: debug auth = 20 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon data avail warn = 5 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon mgr mkfs grace = 240 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon osd reporter subtree level = osd 2026-02-24T18:32:00.533 INFO:teuthology.orchestra.run.trial018.stdout: mon osd prime pg temp = true 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: # 1m isn't quite enough 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon_down_mkfs_grace = 2m 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon_warn_on_filestore_osds = false 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon min osdmap epochs = 50 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon osd crush smoke test = False 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon osdmap full prune interval = 2 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon osdmap full prune min = 15 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon osdmap full prune txsize = 2 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon scrub interval = 300 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: mon warn on pool no app = False 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: paxos service trim min = 10 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout:[client] 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: rgw cache enabled = true 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: rgw enable ops log = true 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: rgw enable usage log = true 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: debug monc = 20 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: debug ms = 1 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: debug objecter = 20 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout: debug rados = 20 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout:[mon.a] 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout:[mon.b] 2026-02-24T18:32:00.534 INFO:teuthology.orchestra.run.trial018.stdout:[mon.c] 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout:[global] 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: chdir = "" 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: auth supported = cephx 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: filestore xattr use omap = true 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: mon clock drift allowed = 1.000 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: osd crush chooseleaf type = 0 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: auth debug = true 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.535 INFO:teuthology.orchestra.run.trial032.stdout: ms die on old message = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: ms die on bug = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon pg warn max object skew = 0 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd pool default size = 2 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon osd allow primary affinity = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon osd allow pg remap = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on legacy crush tunables = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on crush straw calc version zero = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on no sortbitwise = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on osd down out interval zero = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on too few osds = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon_allow_pool_size_one = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd default data pool replay window = 5 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon allow pool delete = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon cluster log file level = debug 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: debug asserts on shutdown = true 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon health detail to clog = false 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon host = "10.20.193.18,10.20.193.32,10.20.193.110" 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon client directed command retry = 5 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: mon election default strategy = 1 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: ms inject socket failures = 5000 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: ms type = async 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd heartbeat grace = 160 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: osd op thread timeout = 150 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: fsid = 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.536 INFO:teuthology.orchestra.run.trial032.stdout:[osd] 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd journal size = 100 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd scrub load threshold = 5.0 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd scrub max interval = 600 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd mclock profile = high_recovery_ops 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd mclock skip benchmark = True 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd recover clone overlap = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd recovery max chunk = 1048576 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug shutdown = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug op order = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug verify stray on activate = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug trim objects = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd open classes on start = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug pg log writeout = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd deep scrub update digest min age = 30 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd map max advance = 10 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: journal zero on create = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: filestore ondisk finisher threads = 3 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: filestore apply finisher threads = 3 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bdev debug aio = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: osd debug misdirected ops = true 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bdev async discard = True 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bdev enable discard = True 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bluestore block size = 96636764160 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bluestore fsck on mount = True 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: bluestore zero block detection = True 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug bluefs = 1 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug bluestore = 1 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug monc = 20 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug ms = 1 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug osd = 20 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: debug rocksdb = 10 2026-02-24T18:32:00.537 INFO:teuthology.orchestra.run.trial032.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon osd full ratio = 0.9 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon osd nearfull ratio = 0.8 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd class default list = * 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd class load list = * 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd client watch timeout = 240 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd debug verify cached snaps = True 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd debug verify missing on start = True 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd failsafe full ratio = 0.95 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd fast shutdown = False 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd mclock override recovery settings = True 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd memory target = 1610612736 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd objectstore = bluestore 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd op queue = debug_random 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd op queue cut off = debug_random 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: osd shutdown pgref assert = True 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout:[mgr] 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug ms = 1 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug mgr = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug mon = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug auth = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mgr/telemetry/nag = false 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout:[mon] 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug ms = 1 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug mon = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug paxos = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: debug auth = 20 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon data avail warn = 5 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon mgr mkfs grace = 240 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon osd reporter subtree level = osd 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon osd prime pg temp = true 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T18:32:00.538 INFO:teuthology.orchestra.run.trial032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: # 1m isn't quite enough 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon_down_mkfs_grace = 2m 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon_warn_on_filestore_osds = false 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon min osdmap epochs = 50 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon osd crush smoke test = False 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon osdmap full prune interval = 2 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon osdmap full prune min = 15 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon osdmap full prune txsize = 2 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon scrub interval = 300 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: mon warn on pool no app = False 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: paxos service trim min = 10 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout:[client] 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: rgw cache enabled = true 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: rgw enable ops log = true 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: rgw enable usage log = true 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: debug monc = 20 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: debug ms = 1 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: debug objecter = 20 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout: debug rados = 20 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout:[mon.a] 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout:[mon.b] 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial032.stdout:[mon.c] 2026-02-24T18:32:00.539 INFO:teuthology.orchestra.run.trial110.stdout:[global] 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: chdir = "" 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: auth supported = cephx 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: filestore xattr use omap = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon clock drift allowed = 1.000 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: osd crush chooseleaf type = 0 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: auth debug = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: ms die on old message = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: ms die on bug = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon pg warn max object skew = 0 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: osd pool default size = 2 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon osd allow primary affinity = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon osd allow pg remap = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on legacy crush tunables = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on crush straw calc version zero = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on no sortbitwise = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on osd down out interval zero = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on too few osds = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon_allow_pool_size_one = true 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: osd default data pool replay window = 5 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.540 INFO:teuthology.orchestra.run.trial110.stdout: mon allow pool delete = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: mon cluster log file level = debug 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: debug asserts on shutdown = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: mon health detail to clog = false 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: mon host = "10.20.193.18,10.20.193.32,10.20.193.110" 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: mon client directed command retry = 5 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: mon election default strategy = 1 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: ms inject socket failures = 5000 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: ms type = async 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd heartbeat grace = 160 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd op thread timeout = 150 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: fsid = 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout:[osd] 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd journal size = 100 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd scrub load threshold = 5.0 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd scrub max interval = 600 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd mclock profile = high_recovery_ops 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd mclock skip benchmark = True 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd recover clone overlap = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd recovery max chunk = 1048576 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd debug shutdown = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd debug op order = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd debug verify stray on activate = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd debug trim objects = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd open classes on start = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd debug pg log writeout = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd deep scrub update digest min age = 30 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: osd map max advance = 10 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: journal zero on create = true 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: filestore ondisk finisher threads = 3 2026-02-24T18:32:00.541 INFO:teuthology.orchestra.run.trial110.stdout: filestore apply finisher threads = 3 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bdev debug aio = true 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd debug misdirected ops = true 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bdev async discard = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bdev enable discard = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bluestore block size = 96636764160 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bluestore fsck on mount = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: bluestore zero block detection = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug bluefs = 1 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug bluestore = 1 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug monc = 20 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug osd = 20 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug rocksdb = 10 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mon osd full ratio = 0.9 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mon osd nearfull ratio = 0.8 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd class default list = * 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd class load list = * 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd client watch timeout = 240 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd debug verify cached snaps = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd debug verify missing on start = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd failsafe full ratio = 0.95 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd fast shutdown = False 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd mclock override recovery settings = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd memory target = 1610612736 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd objectstore = bluestore 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd op queue = debug_random 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd op queue cut off = debug_random 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: osd shutdown pgref assert = True 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout:[mgr] 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug mgr = 20 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug mon = 20 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug auth = 20 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: mgr/telemetry/nag = false 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout:[mon] 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T18:32:00.542 INFO:teuthology.orchestra.run.trial110.stdout: debug mon = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug paxos = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug auth = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon data avail warn = 5 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon mgr mkfs grace = 240 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osd reporter subtree level = osd 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osd prime pg temp = true 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: # 1m isn't quite enough 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon_down_mkfs_grace = 2m 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_filestore_osds = false 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon min osdmap epochs = 50 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osd crush smoke test = False 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osdmap full prune interval = 2 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osdmap full prune min = 15 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon osdmap full prune txsize = 2 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon scrub interval = 300 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on pool no app = False 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: paxos service trim min = 10 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout:[client] 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: rgw cache enabled = true 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: rgw enable ops log = true 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: rgw enable usage log = true 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug monc = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug objecter = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout: debug rados = 20 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout:[mon.a] 2026-02-24T18:32:00.543 INFO:teuthology.orchestra.run.trial110.stdout:[mon.b] 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial110.stdout:[mon.c] 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout:[global] 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: chdir = "" 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: auth supported = cephx 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: filestore xattr use omap = true 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: mon clock drift allowed = 1.000 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: osd crush chooseleaf type = 0 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: auth debug = true 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: ms die on old message = true 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: ms die on bug = true 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: mon pg warn max object skew = 0 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T18:32:00.544 INFO:teuthology.orchestra.run.trial112.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd pool default size = 2 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon osd allow primary affinity = true 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon osd allow pg remap = true 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on legacy crush tunables = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on crush straw calc version zero = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on no sortbitwise = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on osd down out interval zero = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on too few osds = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon_allow_pool_size_one = true 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd default data pool replay window = 5 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon allow pool delete = true 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon cluster log file level = debug 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: debug asserts on shutdown = true 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon health detail to clog = false 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon host = "10.20.193.18,10.20.193.32,10.20.193.110" 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon client directed command retry = 5 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: mon election default strategy = 1 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: ms inject socket failures = 5000 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: ms type = async 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd heartbeat grace = 160 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd op thread timeout = 150 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: fsid = 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout:[osd] 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd journal size = 100 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd scrub load threshold = 5.0 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd scrub max interval = 600 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd mclock profile = high_recovery_ops 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: osd mclock skip benchmark = True 2026-02-24T18:32:00.545 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd recover clone overlap = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd recovery max chunk = 1048576 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug shutdown = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug op order = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug verify stray on activate = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug trim objects = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd open classes on start = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug pg log writeout = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd deep scrub update digest min age = 30 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd map max advance = 10 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: journal zero on create = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: filestore ondisk finisher threads = 3 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: filestore apply finisher threads = 3 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bdev debug aio = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug misdirected ops = true 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bdev async discard = True 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bdev enable discard = True 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bluestore block size = 96636764160 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bluestore fsck on mount = True 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: bluestore zero block detection = True 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug bluefs = 1 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug bluestore = 1 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug monc = 20 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug ms = 1 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug osd = 20 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: debug rocksdb = 10 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: mon osd full ratio = 0.9 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: mon osd nearfull ratio = 0.8 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd class default list = * 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd class load list = * 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd client watch timeout = 240 2026-02-24T18:32:00.546 INFO:teuthology.orchestra.run.trial112.stdout: osd debug verify cached snaps = True 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd debug verify missing on start = True 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd failsafe full ratio = 0.95 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd fast shutdown = False 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd mclock override recovery settings = True 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd memory target = 1610612736 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd objectstore = bluestore 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd op queue = debug_random 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd op queue cut off = debug_random 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: osd shutdown pgref assert = True 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout:[mgr] 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug ms = 1 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug mgr = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug mon = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug auth = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mgr/telemetry/nag = false 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout:[mon] 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug ms = 1 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug mon = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug paxos = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: debug auth = 20 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon data avail warn = 5 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon mgr mkfs grace = 240 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon osd reporter subtree level = osd 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon osd prime pg temp = true 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: # 1m isn't quite enough 2026-02-24T18:32:00.547 INFO:teuthology.orchestra.run.trial112.stdout: mon_down_mkfs_grace = 2m 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon_warn_on_filestore_osds = false 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon min osdmap epochs = 50 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon osd crush smoke test = False 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon osdmap full prune interval = 2 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon osdmap full prune min = 15 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon osdmap full prune txsize = 2 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon scrub interval = 300 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: mon warn on pool no app = False 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: paxos service trim min = 10 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout:[client] 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: rgw cache enabled = true 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: rgw enable ops log = true 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: rgw enable usage log = true 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: debug monc = 20 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: debug ms = 1 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: debug objecter = 20 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout: debug rados = 20 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout:[mon.a] 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout:[mon.b] 2026-02-24T18:32:00.548 INFO:teuthology.orchestra.run.trial112.stdout:[mon.c] 2026-02-24T18:32:00.549 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-24T18:32:00.549 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-24T18:32:00.625 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-24T18:32:00.625 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:32:00.625 DEBUG:teuthology.orchestra.run.trial018:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-24T18:32:00.681 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:32:00.681 DEBUG:teuthology.orchestra.run.trial018:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-24T18:32:00.737 INFO:tasks.ceph:Sending monmap to node ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:32:00.737 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:32:00.737 DEBUG:teuthology.orchestra.run.trial018:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T18:32:00.737 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:32:00.810 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:32:00.810 DEBUG:teuthology.orchestra.run.trial018:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:00.867 INFO:tasks.ceph:Sending monmap to node ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:32:00.867 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:32:00.867 DEBUG:teuthology.orchestra.run.trial032:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T18:32:00.867 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:32:00.900 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:32:00.900 DEBUG:teuthology.orchestra.run.trial032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:00.956 INFO:tasks.ceph:Sending monmap to node ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:32:00.956 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:32:00.956 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T18:32:00.956 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:32:00.989 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:32:00.989 DEBUG:teuthology.orchestra.run.trial110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:01.044 INFO:tasks.ceph:Sending monmap to node ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:32:01.045 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:32:01.045 DEBUG:teuthology.orchestra.run.trial112:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T18:32:01.045 DEBUG:teuthology.orchestra.run.trial112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:32:01.078 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:32:01.078 DEBUG:teuthology.orchestra.run.trial112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:01.133 INFO:tasks.ceph:Setting up mon nodes... 2026-02-24T18:32:01.133 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-24T18:32:01.133 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T18:32:01.178 INFO:teuthology.orchestra.run.trial018.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T18:32:01.180 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T18:32:01.224 INFO:teuthology.orchestra.run.trial112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T18:32:01.226 INFO:tasks.ceph:Setting up mds nodes... 2026-02-24T18:32:01.226 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-24T18:32:01.226 DEBUG:teuthology.orchestra.run.trial112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-24T18:32:01.260 INFO:teuthology.orchestra.run.trial112.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-24T18:32:01.271 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-24T18:32:01.271 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial018.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial032.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial110.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial112.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-24T18:32:01.271 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-24T18:32:01.296 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:01.296 INFO:tasks.ceph:role: osd.0 2026-02-24T18:32:01.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:32:01.297 DEBUG:teuthology.orchestra.run.trial018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:01.361 INFO:teuthology.orchestra.run.trial018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:01.362 INFO:teuthology.orchestra.run.trial018.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:01.362 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:01.362 INFO:teuthology.orchestra.run.trial018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:01.932 INFO:teuthology.orchestra.run.trial018.stdout:Discarding blocks...Done. 2026-02-24T18:32:01.934 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial018.front.sepia.ceph.com -o noatime 2026-02-24T18:32:01.934 DEBUG:teuthology.orchestra.run.trial018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-24T18:32:02.003 DEBUG:teuthology.orchestra.run.trial018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-24T18:32:02.070 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-24T18:32:02.132 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:02.132 INFO:tasks.ceph:role: osd.4 2026-02-24T18:32:02.132 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:32:02.132 DEBUG:teuthology.orchestra.run.trial018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T18:32:02.197 INFO:teuthology.orchestra.run.trial018.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:02.197 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:02.197 INFO:teuthology.orchestra.run.trial018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:02.197 INFO:teuthology.orchestra.run.trial018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:02.197 INFO:teuthology.orchestra.run.trial018.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:02.198 INFO:teuthology.orchestra.run.trial018.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:02.198 INFO:teuthology.orchestra.run.trial018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:02.198 INFO:teuthology.orchestra.run.trial018.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:02.198 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:02.198 INFO:teuthology.orchestra.run.trial018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:02.757 INFO:teuthology.orchestra.run.trial018.stdout:Discarding blocks...Done. 2026-02-24T18:32:02.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial018.front.sepia.ceph.com -o noatime 2026-02-24T18:32:02.759 DEBUG:teuthology.orchestra.run.trial018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-24T18:32:02.826 DEBUG:teuthology.orchestra.run.trial018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-24T18:32:02.893 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-24T18:32:02.958 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:02.958 INFO:tasks.ceph:role: osd.8 2026-02-24T18:32:02.958 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:32:02.958 DEBUG:teuthology.orchestra.run.trial018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T18:32:03.022 INFO:teuthology.orchestra.run.trial018.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:03.022 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:03.022 INFO:teuthology.orchestra.run.trial018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:03.023 INFO:teuthology.orchestra.run.trial018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:03.579 INFO:teuthology.orchestra.run.trial018.stdout:Discarding blocks...Done. 2026-02-24T18:32:03.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial018.front.sepia.ceph.com -o noatime 2026-02-24T18:32:03.582 DEBUG:teuthology.orchestra.run.trial018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2026-02-24T18:32:03.651 DEBUG:teuthology.orchestra.run.trial018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2026-02-24T18:32:03.719 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2026-02-24T18:32:03.783 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:03.783 INFO:tasks.ceph:role: osd.12 2026-02-24T18:32:03.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:32:03.784 DEBUG:teuthology.orchestra.run.trial018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T18:32:03.848 INFO:teuthology.orchestra.run.trial018.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:03.848 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:03.848 INFO:teuthology.orchestra.run.trial018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:03.849 INFO:teuthology.orchestra.run.trial018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:04.406 INFO:teuthology.orchestra.run.trial018.stdout:Discarding blocks...Done. 2026-02-24T18:32:04.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial018.front.sepia.ceph.com -o noatime 2026-02-24T18:32:04.408 DEBUG:teuthology.orchestra.run.trial018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2026-02-24T18:32:04.486 DEBUG:teuthology.orchestra.run.trial018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2026-02-24T18:32:04.555 DEBUG:teuthology.orchestra.run.trial018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:04.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:04.632+0000 7f5c982b2780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-24T18:32:04.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:04.632+0000 7f5c982b2780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-24T18:32:04.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:04.632+0000 7f5c982b2780 -1 bdev(0x555f39bb9800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:04.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:04.633+0000 7f5c982b2780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-24T18:32:08.515 DEBUG:teuthology.orchestra.run.trial018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-24T18:32:08.581 DEBUG:teuthology.orchestra.run.trial018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:08.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:08.656+0000 7f2dac6c8780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-24T18:32:08.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:08.657+0000 7f2dac6c8780 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-24T18:32:08.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:08.657+0000 7f2dac6c8780 -1 bdev(0x55b1e443b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:08.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:08.657+0000 7f2dac6c8780 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-24T18:32:12.535 DEBUG:teuthology.orchestra.run.trial018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-24T18:32:12.600 DEBUG:teuthology.orchestra.run.trial018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:12.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:12.678+0000 7ffa52951780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-24T18:32:12.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:12.678+0000 7ffa52951780 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-24T18:32:12.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:12.678+0000 7ffa52951780 -1 bdev(0x55cc8837f800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:12.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:12.678+0000 7ffa52951780 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-24T18:32:16.558 DEBUG:teuthology.orchestra.run.trial018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-24T18:32:16.623 DEBUG:teuthology.orchestra.run.trial018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:16.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:16.699+0000 7f463d98d780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2026-02-24T18:32:16.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:16.699+0000 7f463d98d780 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2026-02-24T18:32:16.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:16.699+0000 7f463d98d780 -1 bdev(0x55ae153cb800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:16.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:32:16.699+0000 7f463d98d780 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2026-02-24T18:32:20.576 DEBUG:teuthology.orchestra.run.trial018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2026-02-24T18:32:20.643 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-24T18:32:20.669 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:20.669 INFO:tasks.ceph:role: osd.1 2026-02-24T18:32:20.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:32:20.669 DEBUG:teuthology.orchestra.run.trial032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T18:32:20.733 INFO:teuthology.orchestra.run.trial032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:20.733 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:20.733 INFO:teuthology.orchestra.run.trial032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:20.734 INFO:teuthology.orchestra.run.trial032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:21.303 INFO:teuthology.orchestra.run.trial032.stdout:Discarding blocks...Done. 2026-02-24T18:32:21.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial032.front.sepia.ceph.com -o noatime 2026-02-24T18:32:21.306 DEBUG:teuthology.orchestra.run.trial032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2026-02-24T18:32:21.374 DEBUG:teuthology.orchestra.run.trial032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-24T18:32:21.441 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-24T18:32:21.507 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:21.507 INFO:tasks.ceph:role: osd.5 2026-02-24T18:32:21.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:32:21.507 DEBUG:teuthology.orchestra.run.trial032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T18:32:21.571 INFO:teuthology.orchestra.run.trial032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:21.571 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:21.571 INFO:teuthology.orchestra.run.trial032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:21.572 INFO:teuthology.orchestra.run.trial032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:22.130 INFO:teuthology.orchestra.run.trial032.stdout:Discarding blocks...Done. 2026-02-24T18:32:22.132 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial032.front.sepia.ceph.com -o noatime 2026-02-24T18:32:22.132 DEBUG:teuthology.orchestra.run.trial032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2026-02-24T18:32:22.201 DEBUG:teuthology.orchestra.run.trial032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-24T18:32:22.267 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-24T18:32:22.330 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:22.330 INFO:tasks.ceph:role: osd.9 2026-02-24T18:32:22.330 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:32:22.331 DEBUG:teuthology.orchestra.run.trial032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:22.395 INFO:teuthology.orchestra.run.trial032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:22.396 INFO:teuthology.orchestra.run.trial032.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:22.396 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:22.396 INFO:teuthology.orchestra.run.trial032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:22.957 INFO:teuthology.orchestra.run.trial032.stdout:Discarding blocks...Done. 2026-02-24T18:32:22.959 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial032.front.sepia.ceph.com -o noatime 2026-02-24T18:32:22.960 DEBUG:teuthology.orchestra.run.trial032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2026-02-24T18:32:23.026 DEBUG:teuthology.orchestra.run.trial032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2026-02-24T18:32:23.094 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2026-02-24T18:32:23.158 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:23.158 INFO:tasks.ceph:role: osd.13 2026-02-24T18:32:23.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:32:23.158 DEBUG:teuthology.orchestra.run.trial032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T18:32:23.223 INFO:teuthology.orchestra.run.trial032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:23.223 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:23.223 INFO:teuthology.orchestra.run.trial032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:23.224 INFO:teuthology.orchestra.run.trial032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:23.783 INFO:teuthology.orchestra.run.trial032.stdout:Discarding blocks...Done. 2026-02-24T18:32:23.786 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial032.front.sepia.ceph.com -o noatime 2026-02-24T18:32:23.786 DEBUG:teuthology.orchestra.run.trial032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2026-02-24T18:32:23.869 DEBUG:teuthology.orchestra.run.trial032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2026-02-24T18:32:23.936 DEBUG:teuthology.orchestra.run.trial032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:24.016 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:24.013+0000 7f575fe11780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-24T18:32:24.016 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:24.013+0000 7f575fe11780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-24T18:32:24.016 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:24.013+0000 7f575fe11780 -1 bdev(0x55a878647800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:24.016 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:24.013+0000 7f575fe11780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-24T18:32:27.923 DEBUG:teuthology.orchestra.run.trial032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-24T18:32:27.988 DEBUG:teuthology.orchestra.run.trial032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:28.066 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:28.064+0000 7fd3c3016780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-24T18:32:28.067 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:28.064+0000 7fd3c3016780 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-24T18:32:28.067 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:28.064+0000 7fd3c3016780 -1 bdev(0x5590b399b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:28.067 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:28.064+0000 7fd3c3016780 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-24T18:32:31.958 DEBUG:teuthology.orchestra.run.trial032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-24T18:32:32.023 DEBUG:teuthology.orchestra.run.trial032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:32.102 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:32.099+0000 7fa0c80ed780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-24T18:32:32.102 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:32.099+0000 7fa0c80ed780 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-24T18:32:32.102 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:32.100+0000 7fa0c80ed780 -1 bdev(0x564123ff9800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:32.103 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:32.100+0000 7fa0c80ed780 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-24T18:32:35.995 DEBUG:teuthology.orchestra.run.trial032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-24T18:32:36.060 DEBUG:teuthology.orchestra.run.trial032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:36.138 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:36.135+0000 7fe49be14780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2026-02-24T18:32:36.139 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:36.136+0000 7fe49be14780 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2026-02-24T18:32:36.139 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:36.136+0000 7fe49be14780 -1 bdev(0x55c786af5800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:36.139 INFO:teuthology.orchestra.run.trial032.stderr:2026-02-24T18:32:36.136+0000 7fe49be14780 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2026-02-24T18:32:40.023 DEBUG:teuthology.orchestra.run.trial032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2026-02-24T18:32:40.089 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-24T18:32:40.115 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:40.116 INFO:tasks.ceph:role: osd.2 2026-02-24T18:32:40.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:32:40.116 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T18:32:40.180 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:40.180 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:40.180 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:40.181 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:40.750 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T18:32:40.752 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T18:32:40.753 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-24T18:32:40.826 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-24T18:32:40.895 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-24T18:32:40.959 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:40.959 INFO:tasks.ceph:role: osd.6 2026-02-24T18:32:40.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:32:40.960 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T18:32:41.025 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:41.026 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:41.027 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:41.588 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T18:32:41.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T18:32:41.591 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2026-02-24T18:32:41.659 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2026-02-24T18:32:41.727 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2026-02-24T18:32:41.790 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:41.790 INFO:tasks.ceph:role: osd.10 2026-02-24T18:32:41.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:32:41.790 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T18:32:41.855 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:41.855 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:41.856 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:42.413 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T18:32:42.415 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T18:32:42.415 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2026-02-24T18:32:42.482 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2026-02-24T18:32:42.549 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2026-02-24T18:32:42.613 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:42.613 INFO:tasks.ceph:role: osd.14 2026-02-24T18:32:42.613 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:32:42.613 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T18:32:42.677 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:42.678 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:32:43.235 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T18:32:43.237 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T18:32:43.237 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2026-02-24T18:32:43.307 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2026-02-24T18:32:43.375 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:43.455 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:43.453+0000 7fe69e93f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-24T18:32:43.456 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:43.454+0000 7fe69e93f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-24T18:32:43.456 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:43.454+0000 7fe69e93f780 -1 bdev(0x5587973e7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:43.456 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:43.454+0000 7fe69e93f780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-24T18:32:47.341 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-24T18:32:47.407 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:47.485 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:47.484+0000 7f0724e31780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-24T18:32:47.485 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:47.484+0000 7f0724e31780 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-24T18:32:47.485 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:47.484+0000 7f0724e31780 -1 bdev(0x561ca44c9800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:47.485 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:47.484+0000 7f0724e31780 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-24T18:32:51.362 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-24T18:32:51.429 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:51.509 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:51.508+0000 7f305c282780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2026-02-24T18:32:51.509 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:51.508+0000 7f305c282780 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2026-02-24T18:32:51.509 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:51.508+0000 7f305c282780 -1 bdev(0x56025870b800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:51.509 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:51.509+0000 7f305c282780 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2026-02-24T18:32:55.394 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2026-02-24T18:32:55.460 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:32:55.537 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:55.537+0000 7f16f6cfe780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2026-02-24T18:32:55.538 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:55.537+0000 7f16f6cfe780 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2026-02-24T18:32:55.538 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:55.537+0000 7f16f6cfe780 -1 bdev(0x5644098d9800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2026-02-24T18:32:55.538 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T18:32:55.537+0000 7f16f6cfe780 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2026-02-24T18:32:59.426 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2026-02-24T18:32:59.491 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-24T18:32:59.517 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-24T18:32:59.517 INFO:tasks.ceph:role: osd.3 2026-02-24T18:32:59.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:32:59.518 DEBUG:teuthology.orchestra.run.trial112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T18:32:59.582 INFO:teuthology.orchestra.run.trial112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:32:59.583 INFO:teuthology.orchestra.run.trial112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:33:00.153 INFO:teuthology.orchestra.run.trial112.stdout:Discarding blocks...Done. 2026-02-24T18:33:00.155 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial112.front.sepia.ceph.com -o noatime 2026-02-24T18:33:00.155 DEBUG:teuthology.orchestra.run.trial112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-24T18:33:00.223 DEBUG:teuthology.orchestra.run.trial112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-24T18:33:00.299 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-24T18:33:00.364 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-24T18:33:00.364 INFO:tasks.ceph:role: osd.7 2026-02-24T18:33:00.365 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:33:00.365 DEBUG:teuthology.orchestra.run.trial112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T18:33:00.429 INFO:teuthology.orchestra.run.trial112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:33:00.430 INFO:teuthology.orchestra.run.trial112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:33:00.992 INFO:teuthology.orchestra.run.trial112.stdout:Discarding blocks...Done. 2026-02-24T18:33:00.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial112.front.sepia.ceph.com -o noatime 2026-02-24T18:33:00.995 DEBUG:teuthology.orchestra.run.trial112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2026-02-24T18:33:01.063 DEBUG:teuthology.orchestra.run.trial112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2026-02-24T18:33:01.134 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2026-02-24T18:33:01.198 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-24T18:33:01.199 INFO:tasks.ceph:role: osd.11 2026-02-24T18:33:01.199 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:33:01.199 DEBUG:teuthology.orchestra.run.trial112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:33:01.264 INFO:teuthology.orchestra.run.trial112.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:33:01.265 INFO:teuthology.orchestra.run.trial112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:33:01.265 INFO:teuthology.orchestra.run.trial112.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:33:01.265 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:33:01.265 INFO:teuthology.orchestra.run.trial112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:33:01.824 INFO:teuthology.orchestra.run.trial112.stdout:Discarding blocks...Done. 2026-02-24T18:33:01.826 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial112.front.sepia.ceph.com -o noatime 2026-02-24T18:33:01.826 DEBUG:teuthology.orchestra.run.trial112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2026-02-24T18:33:01.898 DEBUG:teuthology.orchestra.run.trial112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2026-02-24T18:33:01.968 DEBUG:teuthology.orchestra.run.trial112:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2026-02-24T18:33:02.032 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-24T18:33:02.032 INFO:tasks.ceph:role: osd.15 2026-02-24T18:33:02.032 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:33:02.033 DEBUG:teuthology.orchestra.run.trial112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T18:33:02.097 INFO:teuthology.orchestra.run.trial112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:33:02.097 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:33:02.097 INFO:teuthology.orchestra.run.trial112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:33:02.097 INFO:teuthology.orchestra.run.trial112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:33:02.097 INFO:teuthology.orchestra.run.trial112.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:33:02.098 INFO:teuthology.orchestra.run.trial112.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:33:02.098 INFO:teuthology.orchestra.run.trial112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:33:02.098 INFO:teuthology.orchestra.run.trial112.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:33:02.098 INFO:teuthology.orchestra.run.trial112.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:33:02.098 INFO:teuthology.orchestra.run.trial112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:33:02.657 INFO:teuthology.orchestra.run.trial112.stdout:Discarding blocks...Done. 2026-02-24T18:33:02.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial112.front.sepia.ceph.com -o noatime 2026-02-24T18:33:02.659 DEBUG:teuthology.orchestra.run.trial112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2026-02-24T18:33:02.736 DEBUG:teuthology.orchestra.run.trial112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2026-02-24T18:33:02.805 DEBUG:teuthology.orchestra.run.trial112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:02.883 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:02.880+0000 7fd28780f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-24T18:33:02.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:02.880+0000 7fd28780f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-24T18:33:02.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:02.880+0000 7fd28780f780 -1 bdev(0x55d72be77800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-24T18:33:02.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:02.880+0000 7fd28780f780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-24T18:33:06.765 DEBUG:teuthology.orchestra.run.trial112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-24T18:33:06.831 DEBUG:teuthology.orchestra.run.trial112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:06.908 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:06.905+0000 7f91565d4780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-24T18:33:06.909 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:06.906+0000 7f91565d4780 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-24T18:33:06.909 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:06.906+0000 7f91565d4780 -1 bdev(0x55f05cd9f800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-24T18:33:06.909 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:06.906+0000 7f91565d4780 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-24T18:33:10.787 DEBUG:teuthology.orchestra.run.trial112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-24T18:33:10.851 DEBUG:teuthology.orchestra.run.trial112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:10.929 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:10.926+0000 7f358be9d780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2026-02-24T18:33:10.929 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:10.927+0000 7f358be9d780 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2026-02-24T18:33:10.929 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:10.927+0000 7f358be9d780 -1 bdev(0x555eec7a9800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2026-02-24T18:33:10.929 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:10.927+0000 7f358be9d780 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2026-02-24T18:33:14.815 DEBUG:teuthology.orchestra.run.trial112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2026-02-24T18:33:14.881 DEBUG:teuthology.orchestra.run.trial112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:14.960 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:14.957+0000 7f3a99e13780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2026-02-24T18:33:14.960 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:14.957+0000 7f3a99e13780 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2026-02-24T18:33:14.960 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:14.958+0000 7f3a99e13780 -1 bdev(0x559c4eb97800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2026-02-24T18:33:14.960 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:14.958+0000 7f3a99e13780 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2026-02-24T18:33:18.837 DEBUG:teuthology.orchestra.run.trial112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2026-02-24T18:33:18.902 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-24T18:33:18.903 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:18.903 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-24T18:33:18.929 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:18.929 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-24T18:33:18.993 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:18.993 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-24T18:33:19.059 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:19.060 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-24T18:33:19.125 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:19.125 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2026-02-24T18:33:19.189 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:19.189 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-24T18:33:19.215 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:19.215 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-24T18:33:19.279 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:19.280 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-24T18:33:19.343 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:19.343 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2026-02-24T18:33:19.405 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:19.405 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-24T18:33:19.431 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:19.431 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-24T18:33:19.496 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:19.497 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2026-02-24T18:33:19.560 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:19.560 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2026-02-24T18:33:19.625 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.626 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-24T18:33:19.651 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.651 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-24T18:33:19.715 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.715 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-24T18:33:19.778 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.778 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2026-02-24T18:33:19.842 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.842 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2026-02-24T18:33:19.905 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:19.906 DEBUG:teuthology.orchestra.run.trial112:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-24T18:33:19.961 INFO:tasks.ceph:Adding keys to all mons... 2026-02-24T18:33:19.962 DEBUG:teuthology.orchestra.run.trial018:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T18:33:19.964 DEBUG:teuthology.orchestra.run.trial032:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T18:33:19.967 DEBUG:teuthology.orchestra.run.trial110:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T18:33:19.986 INFO:teuthology.orchestra.run.trial018.stdout:[mgr.y] 2026-02-24T18:33:19.986 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCh7p1pOxKZChAAs198X2/gHJUEQKR2B/cQrw== 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout:[osd.0] 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCk7p1pwbbCJRAATWSocUKDXJe3l4D3T4uOEg== 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout:[osd.4] 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCo7p1pnsxAJxAAqHCA5KyH4X4MPZv2y1QfKw== 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout:[osd.8] 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCs7p1pohx4KBAAsBEcMEqL2rb4hfiu/bo0hw== 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout:[osd.12] 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCw7p1pjRy+KRAA6f1eWQ+3vdu3lQ9EYuCwZg== 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout:[osd.1] 2026-02-24T18:33:19.987 INFO:teuthology.orchestra.run.trial018.stdout: key = AQC47p1peU7YABAAX0ux8LuDT/m4qMswSVJPfA== 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout:[osd.5] 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout: key = AQC87p1pc2XdAxAAoXmwHjPTLch2i1egqhKe0w== 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout:[osd.9] 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDA7p1pPUb9BRAAN/eCKMN99SIen7tTVS/CYg== 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout:[osd.13] 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDE7p1p3R0kCBAAZxQhBHZ1AuUTdKyLrLbA4w== 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout:[osd.2] 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDL7p1pRFgmGxAAgAotbhj31co2Y6/zzAsrvQ== 2026-02-24T18:33:19.988 INFO:teuthology.orchestra.run.trial018.stdout:[osd.6] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDP7p1psFvlHBAA3DUYDf88yzJFuwQrRQVC0Q== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[osd.10] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDT7p1pdYpaHhAAyKxaIO/b/Ej35mBPL++MWg== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[osd.14] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDX7p1pE+YQIBAAW8Ns5JyoPXcKFgRXTyMgyw== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[mgr.x] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCh7p1pJepADRAAmn2x/8/kis7HeMLv0LJsEw== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[osd.3] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDe7p1pU7eMNBAA6ZsvJyHVMlJI/QUrXXTdYQ== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[osd.7] 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDi7p1p3/0PNhAA7nLPkDNvOY1L1dBjE2vXDg== 2026-02-24T18:33:19.989 INFO:teuthology.orchestra.run.trial018.stdout:[osd.11] 2026-02-24T18:33:19.990 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDm7p1pnK9BNxAA03n66GrXaBhrlE/lrCa1ww== 2026-02-24T18:33:19.990 INFO:teuthology.orchestra.run.trial018.stdout:[osd.15] 2026-02-24T18:33:19.990 INFO:teuthology.orchestra.run.trial018.stdout: key = AQDq7p1p8FAZORAAJO1U5jO/kqVXmLFIHiwj5w== 2026-02-24T18:33:19.990 INFO:teuthology.orchestra.run.trial018.stdout:[client.0] 2026-02-24T18:33:19.990 INFO:teuthology.orchestra.run.trial018.stdout: key = AQCh7p1pnLNrDxAAFGhv3ChrUL0MYDqq3+/lpQ== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[mgr.y] 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCh7p1pOxKZChAAs198X2/gHJUEQKR2B/cQrw== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[osd.0] 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCk7p1pwbbCJRAATWSocUKDXJe3l4D3T4uOEg== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[osd.4] 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCo7p1pnsxAJxAAqHCA5KyH4X4MPZv2y1QfKw== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[osd.8] 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCs7p1pohx4KBAAsBEcMEqL2rb4hfiu/bo0hw== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[osd.12] 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCw7p1pjRy+KRAA6f1eWQ+3vdu3lQ9EYuCwZg== 2026-02-24T18:33:19.991 INFO:teuthology.orchestra.run.trial032.stdout:[osd.1] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQC47p1peU7YABAAX0ux8LuDT/m4qMswSVJPfA== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.5] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQC87p1pc2XdAxAAoXmwHjPTLch2i1egqhKe0w== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.9] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDA7p1pPUb9BRAAN/eCKMN99SIen7tTVS/CYg== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.13] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDE7p1p3R0kCBAAZxQhBHZ1AuUTdKyLrLbA4w== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.2] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDL7p1pRFgmGxAAgAotbhj31co2Y6/zzAsrvQ== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.6] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDP7p1psFvlHBAA3DUYDf88yzJFuwQrRQVC0Q== 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout:[osd.10] 2026-02-24T18:33:19.992 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDT7p1pdYpaHhAAyKxaIO/b/Ej35mBPL++MWg== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[osd.14] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDX7p1pE+YQIBAAW8Ns5JyoPXcKFgRXTyMgyw== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[mgr.x] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCh7p1pJepADRAAmn2x/8/kis7HeMLv0LJsEw== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[osd.3] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDe7p1pU7eMNBAA6ZsvJyHVMlJI/QUrXXTdYQ== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[osd.7] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDi7p1p3/0PNhAA7nLPkDNvOY1L1dBjE2vXDg== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[osd.11] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDm7p1pnK9BNxAA03n66GrXaBhrlE/lrCa1ww== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[osd.15] 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout: key = AQDq7p1p8FAZORAAJO1U5jO/kqVXmLFIHiwj5w== 2026-02-24T18:33:19.993 INFO:teuthology.orchestra.run.trial032.stdout:[client.0] 2026-02-24T18:33:19.994 INFO:teuthology.orchestra.run.trial032.stdout: key = AQCh7p1pnLNrDxAAFGhv3ChrUL0MYDqq3+/lpQ== 2026-02-24T18:33:19.996 INFO:teuthology.orchestra.run.trial110.stdout:[mgr.y] 2026-02-24T18:33:19.996 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCh7p1pOxKZChAAs198X2/gHJUEQKR2B/cQrw== 2026-02-24T18:33:19.996 INFO:teuthology.orchestra.run.trial110.stdout:[osd.0] 2026-02-24T18:33:19.996 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCk7p1pwbbCJRAATWSocUKDXJe3l4D3T4uOEg== 2026-02-24T18:33:19.996 INFO:teuthology.orchestra.run.trial110.stdout:[osd.4] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCo7p1pnsxAJxAAqHCA5KyH4X4MPZv2y1QfKw== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.8] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCs7p1pohx4KBAAsBEcMEqL2rb4hfiu/bo0hw== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.12] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCw7p1pjRy+KRAA6f1eWQ+3vdu3lQ9EYuCwZg== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.1] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQC47p1peU7YABAAX0ux8LuDT/m4qMswSVJPfA== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.5] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQC87p1pc2XdAxAAoXmwHjPTLch2i1egqhKe0w== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.9] 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDA7p1pPUb9BRAAN/eCKMN99SIen7tTVS/CYg== 2026-02-24T18:33:19.997 INFO:teuthology.orchestra.run.trial110.stdout:[osd.13] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDE7p1p3R0kCBAAZxQhBHZ1AuUTdKyLrLbA4w== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.2] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDL7p1pRFgmGxAAgAotbhj31co2Y6/zzAsrvQ== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.6] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDP7p1psFvlHBAA3DUYDf88yzJFuwQrRQVC0Q== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.10] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDT7p1pdYpaHhAAyKxaIO/b/Ej35mBPL++MWg== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.14] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDX7p1pE+YQIBAAW8Ns5JyoPXcKFgRXTyMgyw== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[mgr.x] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCh7p1pJepADRAAmn2x/8/kis7HeMLv0LJsEw== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.3] 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDe7p1pU7eMNBAA6ZsvJyHVMlJI/QUrXXTdYQ== 2026-02-24T18:33:19.998 INFO:teuthology.orchestra.run.trial110.stdout:[osd.7] 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDi7p1p3/0PNhAA7nLPkDNvOY1L1dBjE2vXDg== 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout:[osd.11] 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDm7p1pnK9BNxAA03n66GrXaBhrlE/lrCa1ww== 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout:[osd.15] 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout: key = AQDq7p1p8FAZORAAJO1U5jO/kqVXmLFIHiwj5w== 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout:[client.0] 2026-02-24T18:33:19.999 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCh7p1pnLNrDxAAFGhv3ChrUL0MYDqq3+/lpQ== 2026-02-24T18:33:20.000 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:20.037 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:20.039 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:20.080 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.116 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.120 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.160 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.197 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.200 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.240 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.275 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.278 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.321 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.354 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.357 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.400 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.434 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.437 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.480 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.514 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.517 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.559 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.592 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.596 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.637 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.673 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.676 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.716 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.752 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.755 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.795 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.832 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.834 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.874 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.911 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.914 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.954 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.991 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:20.995 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.037 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:21.070 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:21.076 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:33:21.117 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.149 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.156 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.196 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.227 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.234 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.276 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.306 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.313 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.354 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.385 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.393 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:33:21.435 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T18:33:21.463 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T18:33:21.474 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T18:33:21.517 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-24T18:33:21.517 DEBUG:teuthology.orchestra.run.trial018:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-24T18:33:21.564 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:33:21.651 DEBUG:teuthology.orchestra.run.trial018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-24T18:33:21.675 DEBUG:teuthology.orchestra.run.trial032:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-24T18:33:21.700 DEBUG:teuthology.orchestra.run.trial032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:33:21.787 DEBUG:teuthology.orchestra.run.trial032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-24T18:33:21.810 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-24T18:33:21.835 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:33:21.921 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-24T18:33:21.944 DEBUG:teuthology.orchestra.run.trial018:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:21.946 DEBUG:teuthology.orchestra.run.trial032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:21.948 DEBUG:teuthology.orchestra.run.trial110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:33:22.000 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-24T18:33:22.000 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:33:22.001 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-24T18:33:22.001 DEBUG:teuthology.orchestra.run.trial018:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2026-02-24T18:33:22.003 INFO:tasks.ceph.mon.a:Started 2026-02-24T18:33:22.003 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:33:22.003 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-24T18:33:22.003 DEBUG:teuthology.orchestra.run.trial032:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2026-02-24T18:33:22.005 INFO:tasks.ceph.mon.b:Started 2026-02-24T18:33:22.005 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:33:22.005 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-24T18:33:22.005 DEBUG:teuthology.orchestra.run.trial110:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2026-02-24T18:33:22.042 INFO:tasks.ceph.mon.c:Started 2026-02-24T18:33:22.042 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-24T18:33:22.042 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-24T18:33:22.042 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2026-02-24T18:33:22.044 INFO:tasks.ceph.mgr.y:Started 2026-02-24T18:33:22.044 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-24T18:33:22.044 DEBUG:teuthology.orchestra.run.trial112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2026-02-24T18:33:22.046 INFO:tasks.ceph.mgr.x:Started 2026-02-24T18:33:22.046 INFO:tasks.ceph:Setting crush tunables to default 2026-02-24T18:33:22.046 DEBUG:teuthology.orchestra.run.trial018:> sudo ceph --cluster ceph osd crush tunables default 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.110+0000 7f08f85e4640 10 monclient: get_monmap_and_config 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.110+0000 7f08f85e4640 10 monclient: build_initial_monmap 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.110+0000 7f08f85e4640 10 monclient: monmap: 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:33:22.111744+0000 2026-02-24T18:33:22.112 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:33:22.111744+0000 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:22.113 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 Processor -- start 2026-02-24T18:33:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 -- start start 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient: init 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient: _add_conns ranks=[0,5,1,4,3,2] 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08f01527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-c con 0x7f08f0073530 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f08f015f900 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f00739a0 con 0x7f08f01527f0 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f015fd90 con 0x7f08f015f900 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08f0073530 2026-02-24T18:33:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f85e4640 10 monclient(hunting): authenticate will time out at 1345.792959s 2026-02-24T18:33:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.111+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08f01527f0 legacy=0x7f08f015d2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.116+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 msgr2=0x7f08f015f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.116+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:22.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.132+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08f015f900 legacy=0x7f08f00ccf70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:22.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.312+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08f01527f0 legacy=0x7f08f015d2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:22.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.317+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 msgr2=0x7f08f015f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:22.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.317+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:22.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.333+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08f015f900 legacy=0x7f08f00ccf70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:22.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.713+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08f01527f0 legacy=0x7f08f015d2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:22.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.718+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 msgr2=0x7f08f015f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:22.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.718+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:22.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:22.734+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08f015f900 legacy=0x7f08f00ccf70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:23.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:23.514+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08f01527f0 legacy=0x7f08f015d2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:23.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:23.519+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 msgr2=0x7f08f015f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:23.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:23.519+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:23.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:23.535+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08f015f900 legacy=0x7f08f00ccf70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 msgr2=0x7f08f015f3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08f0073530 0x7f08f015f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08f015f900 legacy=0x7f08f00ccf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:25.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08f01527f0 legacy=0x7f08f015d2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 10 monclient: _add_conns ranks=[3,2,4] 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f08d00037d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f08d0006270 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08f6359640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 msgr2=0x7f08d0005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08f6359640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f08d000a000 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f015d9d0 con 0x7f08d0006270 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:25.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d00037d0 2026-02-24T18:33:25.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:25.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08d000dbe0 con 0x7f08d000a000 2026-02-24T18:33:25.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:25.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08f6b5a640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:25.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.112+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:25.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.313+0000 7f08f6359640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 msgr2=0x7f08d0005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:25.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.313+0000 7f08f6359640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:25.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.313+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:25.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.313+0000 7f08f6b5a640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:25.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.313+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:25.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.714+0000 7f08f6359640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 msgr2=0x7f08d0005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:25.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.714+0000 7f08f6359640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:25.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.714+0000 7f08f6b5a640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:25.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.714+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:25.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:25.714+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:26.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:26.515+0000 7f08f6359640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 msgr2=0x7f08d0005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:26.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:26.515+0000 7f08f6359640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:26.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:26.515+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:26.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:26.515+0000 7f08f6b5a640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:26.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:26.515+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000a000 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 msgr2=0x7f08d0005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d00037d0 0x7f08d0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:28.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08f6b5a640 1 -- reap_dead start 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient: _add_conns ranks=[5,2,1,3,0,4] 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f08d000cba0 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f08d0006270 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f08d000d730 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d000ddf0 con 0x7f08d000d730 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f015d9d0 con 0x7f08d0006270 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d000cba0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d000d020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08f6359640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08f6359640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:28.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.111+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d000d730 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:28.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.312+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d000d730 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:28.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.312+0000 7f08f6359640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:28.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.312+0000 7f08f6359640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:28.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.312+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d000d020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:28.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.713+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d000d020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:28.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.713+0000 7f08f6359640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:28.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.713+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d000d730 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:28.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:28.713+0000 7f08f6359640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:29.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:29.514+0000 7f08f5b58640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d000d020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:29.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:29.514+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d000d730 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:29.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:29.514+0000 7f08f6359640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:29.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:29.514+0000 7f08f6359640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:31.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:31.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 msgr2=0x7f08d000c4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000cba0 0x7f08d000c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0006270 legacy=0x7f08d000d020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d000d730 legacy=0x7f08d0008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient: _add_conns ranks=[4,3,0] 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f08d000f1b0 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:31.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f08d0011ae0 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08d0014500 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d0005ca0 con 0x7f08d0014500 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d0011ae0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08d000dbe0 con 0x7f08d000f1b0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08f5b58640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08f5b58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08f6b5a640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d0014500 legacy=0x7f08d00169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08f6359640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 msgr2=0x7f08d00115a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:31.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.111+0000 7f08f6359640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:31.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.312+0000 7f08f6b5a640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d0014500 legacy=0x7f08d00169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:31.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.312+0000 7f08f5b58640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:31.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.312+0000 7f08f5b58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:31.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.312+0000 7f08f6359640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 msgr2=0x7f08d00115a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:31.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.312+0000 7f08f6359640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:31.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.713+0000 7f08f6b5a640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d0014500 legacy=0x7f08d00169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:31.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.713+0000 7f08f5b58640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:31.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.713+0000 7f08f5b58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:31.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.713+0000 7f08f6359640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 msgr2=0x7f08d00115a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:31.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:31.713+0000 7f08f6359640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:32.481 INFO:tasks.ceph.mon.a.trial018.stderr:--00:00:00:10.426 45353-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:33:32.503 INFO:tasks.ceph.mon.c.trial110.stderr:--00:00:00:10.411 45171-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:33:32.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:32.514+0000 7f08f6359640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 msgr2=0x7f08d00115a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:32.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:32.514+0000 7f08f6359640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:32.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:32.514+0000 7f08f6b5a640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d0014500 legacy=0x7f08d00169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:32.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:32.514+0000 7f08f5b58640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:32.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:32.514+0000 7f08f5b58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:32.632 INFO:tasks.ceph.mon.b.trial032.stderr:--00:00:00:10.577 44572-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 msgr2=0x7f08d00115a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000f1b0 0x7f08d00115a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:34.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0011ae0 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d0014500 legacy=0x7f08d00169a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f6b5a640 1 -- reap_dead start 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f6b5a640 1 -- reap_dead start 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient: _add_conns ranks=[2,5,3,0,4,1] 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f08d0014500 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f08d0006270 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f08d000f590 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d000ddf0 con 0x7f08d0014500 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d000f590 2026-02-24T18:33:34.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f08d000dbe0 con 0x7f08d0006270 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f6b5a640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f6359640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0014500 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:34.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.112+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.313+0000 7f08f6b5a640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.313+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.313+0000 7f08f6359640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0014500 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.313+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.313+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:34.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.714+0000 7f08f6b5a640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:34.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.714+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:34.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.714+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:34.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.714+0000 7f08f6359640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0014500 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:34.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:34.714+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:35.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:35.515+0000 7f08f6b5a640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.18:3300/0 2026-02-24T18:33:35.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:35.515+0000 7f08f6b5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:35.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:35.515+0000 7f08f6359640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0014500 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.110:6789/0 2026-02-24T18:33:35.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:35.515+0000 7f08f5b58640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.110:3300/0 2026-02-24T18:33:35.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:35.515+0000 7f08f5b58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000e8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d0006270 0x7f08d000e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d000f590 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- >> v1:10.20.193.110:6789/0 conn(0x7f08d0014500 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient: _add_conns ranks=[0,4,1] 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08d000a990 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f08d000c490 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f08d00168f0 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d0005ca0 con 0x7f08d000a990 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d000ddf0 con 0x7f08d00168f0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d000c490 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d000a990 legacy=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08f5b58640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d00168f0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.112+0000 7f08f5b58640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:37.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.313+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d000a990 legacy=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:37.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.313+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d00168f0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:37.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.313+0000 7f08f5b58640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:37.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.313+0000 7f08f5b58640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:37.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.714+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d000a990 legacy=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:37.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.714+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d00168f0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:37.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.714+0000 7f08f5b58640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:37.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:37.714+0000 7f08f5b58640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:38.030 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:00:15.975 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.030 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:00:15.938 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.178 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:00:16.087 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.179 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:00:16.125 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.224 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:00:16.170 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.377 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:00:16.322 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:33:38.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:38.515+0000 7f08f6359640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d000a990 legacy=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.18:6789/0 2026-02-24T18:33:38.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:38.515+0000 7f08f6b5a640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d00168f0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.32:6789/0 2026-02-24T18:33:38.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:38.515+0000 7f08f5b58640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.32:3300/0 2026-02-24T18:33:38.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:38.515+0000 7f08f5b58640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 msgr2=0x7f08d0013ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f08d000c490 0x7f08d0013ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- >> v1:10.20.193.32:6789/0 conn(0x7f08d00168f0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:40.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- >> v1:10.20.193.18:6789/0 conn(0x7f08d000a990 legacy=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08f6b5a640 1 -- reap_dead start 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08f6b5a640 1 -- reap_dead start 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient: _add_conns ranks=[3,1,5,0,2,4] 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f08d0006270 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f08d000f590 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f08d000e8b0 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f015d9d0 con 0x7f08d000f590 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d0006270 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f08d000dbe0 con 0x7f08d000e8b0 2026-02-24T18:33:40.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.112+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:40.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.116+0000 7f08f5b58640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f08d000f590 0x7f08d000e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:39586/0 (socket says 10.20.193.18:39586) 2026-02-24T18:33:40.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.116+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 learned_addr learned my addr 10.20.193.18:0/1099899678 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:33:40.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.119+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d000e8b0 auth_method 0 2026-02-24T18:33:40.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.121+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d0006270 auth_method 0 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.122+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:40.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.322+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.322+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d000e8b0 auth_method 0 2026-02-24T18:33:40.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.322+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.322+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.322+0000 7f08f6b5a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:40.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d0006270 auth_method 0 2026-02-24T18:33:40.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.323+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d000e8b0 auth_method 0 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6b5a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d0006270 auth_method 0 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.724+0000 7f08f6359640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:40.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:40.725+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d000e8b0 auth_method 0 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:41.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d0006270 auth_method 0 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6359640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.526+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.527+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.527+0000 7f08f6359640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.527+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:41.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:41.527+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:43.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): tick 2026-02-24T18:33:43.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 monclient(hunting): continuing hunt 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 msgr2=0x7f08d000ad70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.110:3300/0 conn(0x7f08d000e8b0 0x7f08d000ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 msgr2=0x7f08d000a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.18:3300/0 conn(0x7f08d0006270 0x7f08d000a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 >> v1:10.20.193.32:6789/0 conn(0x7f08d000f590 legacy=0x7f08d000e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient: _add_conns ranks=[0,4,2] 2026-02-24T18:33:43.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08d000b2b0 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f08d0013ed0 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f08d000c2e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d0005ca0 con 0x7f08d000b2b0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d000ddf0 con 0x7f08d000c2e0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 1 -- 10.20.193.18:0/1099899678 --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d0013ed0 2026-02-24T18:33:43.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.113+0000 7f08d7fff640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:43.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.117+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:43.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d0013ed0 auth_method 0 2026-02-24T18:33:43.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:43.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:43.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:43.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-24T18:33:43.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:43.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:43.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.120+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:43.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.321+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:43.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.321+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d0013ed0 auth_method 0 2026-02-24T18:33:43.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.321+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.321+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.321+0000 7f08f5b58640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.322+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.322+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.322+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:43.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.322+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.723+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.723+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d0013ed0 auth_method 0 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.723+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.723+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.723+0000 7f08f5b58640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.724+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.724+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.724+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:43.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:43.724+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-24T18:33:44.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.525+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.525+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d0013ed0 auth_method 0 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.525+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.525+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.525+0000 7f08f5b58640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.526+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-24T18:33:44.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.526+0000 7f08f5b58640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:33:44.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.526+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:44.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:44.526+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-24T18:33:45.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31776278 0 0) 0x7f08d0005ca0 con 0x7f08d000b2b0 2026-02-24T18:33:45.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 10 monclient(hunting): _negotiate 2026-02-24T18:33:45.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:45.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 10 monclient(hunting): my global_id is 4098 2026-02-24T18:33:45.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.454+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08d8003620 con 0x7f08d000b2b0 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1683001593 0 0) 0x7f08d000ddf0 con 0x7f08d000c2e0 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 10 monclient(hunting): _negotiate 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 10 monclient(hunting): my global_id is 4103 2026-02-24T18:33:45.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.470+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08d0005ca0 con 0x7f08d000c2e0 2026-02-24T18:33:45.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382630458 0 0) 0x7f08d8003620 con 0x7f08d000b2b0 2026-02-24T18:33:45.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 msgr2=0x7f08d000bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:33:45.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 --2- 10.20.193.18:0/1099899678 >> v2:10.20.193.32:3300/0 conn(0x7f08d0013ed0 0x7f08d000bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 >> v1:10.20.193.110:6789/0 conn(0x7f08d000c2e0 legacy=0x7f08d0008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:45.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _finish_hunting 0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 reap_dead start 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d00164d0 con 0x7f08d000b2b0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _finish_auth 0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _check_auth_tickets 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f08e0009910 con 0x7f08d000b2b0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:45.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: dump: 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 >> v1:10.20.193.18:6789/0 conn(0x7f08d000b2b0 legacy=0x7f08d000b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8003c90 0x7f08d8004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): picked mon.a con 0x7f08d8003c90 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8004900 0x7f08d8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): picked mon.c con 0x7f08d8004900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 0x7f08d800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): picked mon.b con 0x7f08d8005430 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0074b10 con 0x7f08d8003c90 2026-02-24T18:33:45.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08d800c380 con 0x7f08d8005430 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08d800c580 con 0x7f08d8004900 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.482+0000 7f08f5357640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 0x7f08d800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d8005430 auth_method 0 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f6b5a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f5b58640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8004900 0x7f08d8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d8004900 auth_method 0 2026-02-24T18:33:45.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.483+0000 7f08f5b58640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:45.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.484+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:45.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.484+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:45.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.484+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:45.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.485+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:45.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.485+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:45.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.485+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:45.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.490+0000 7f08f6359640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8003c90 0x7f08d8004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.492+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d8003c90 auth_method 0 2026-02-24T18:33:45.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.492+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.492+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.492+0000 7f08f6359640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:45.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_done global_id 4098 payload 293 2026-02-24T18:33:45.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8004900 msgr2=0x7f08d8004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:45.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8004900 0x7f08d8004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8003c90 msgr2=0x7f08d8004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8003c90 0x7f08d8004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 10 monclient: _finish_hunting 0 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 monclient: found mon.b 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 1 -- 10.20.193.18:0/1099899678 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d8003620 con 0x7f08d8005430 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 10 monclient: _finish_auth 0 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 10 monclient: _check_auth_tickets 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f6b5a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f85e4640 5 monclient: authenticate success, global_id 4098 2026-02-24T18:33:45.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.506+0000 7f08f85e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:45.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.508+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 0x7f08d800bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08f00c88f0 tx=0x7f08ec030800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98ae647d6008663 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:45.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec03d070 con 0x7f08d8005430 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: dump: 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.547 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f85e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec037600 con 0x7f08d8005430 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 1 -- 10.20.193.18:0/1099899678 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec007a90 con 0x7f08d8005430 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:45.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f5357640 10 monclient: dump: 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f85e4640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:45.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.544+0000 7f08f85e4640 4 set_mon_vals no callback set 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 10 monclient: shutdown 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 -- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 msgr2=0x7f08d800bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 0x7f08d800bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08f00c88f0 tx=0x7f08ec030800 comp rx=0 tx=0).stop 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 -- 10.20.193.18:0/1099899678 shutdown_connections 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d8005430 0x7f08d800bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8004900 0x7f08d8004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 --2- 10.20.193.18:0/1099899678 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8003c90 0x7f08d8004090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 -- 10.20.193.18:0/1099899678 >> 10.20.193.18:0/1099899678 conn(0x7f08f00ce730 msgr2=0x7f08f00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 -- 10.20.193.18:0/1099899678 shutdown_connections 2026-02-24T18:33:45.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 1 -- 10.20.193.18:0/1099899678 wait complete. 2026-02-24T18:33:45.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 10 monclient: build_initial_monmap 2026-02-24T18:33:45.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.545+0000 7f08f85e4640 10 monclient: monmap: 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 librados: starting msgr at 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 librados: starting objecter 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 Processor -- start 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 -- start start 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 librados: setting wanted keys 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 librados: calling monclient init 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient: init 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 0x7f08f00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-b con 0x7f08d000b2b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 0x7f08f00c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-c con 0x7f08d8003c90 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): picked mon.noname-a con 0x7f08d8004900 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00747a0 con 0x7f08d8004900 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00744c0 con 0x7f08d000b2b0 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0074fa0 con 0x7f08d8003c90 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.546+0000 7f08f85e4640 10 monclient(hunting): authenticate will time out at 1369.227824s 2026-02-24T18:33:45.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.549+0000 7f08f6b5a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.552+0000 7f08f5b58640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 0x7f08f00c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.552+0000 7f08f6359640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 0x7f08f00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 0x7f08f00c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:45066/0 (socket says 10.20.193.18:45066) 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 1 -- 10.20.193.18:0/142147654 learned_addr learned my addr 10.20.193.18:0/142147654 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 10 monclient(hunting): get_auth_request con 0x7f08d8003c90 auth_method 0 2026-02-24T18:33:45.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f5b58640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request con 0x7f08d8004900 auth_method 0 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6b5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6359640 10 monclient(hunting): get_auth_request con 0x7f08d000b2b0 auth_method 0 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.553+0000 7f08f6359640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:45.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f5b58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6359640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6359640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6359640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.554+0000 7f08f6b5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:45.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.558+0000 7f08f6b5a640 1 -- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 msgr2=0x7f08f00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-24T18:33:45.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.558+0000 7f08f6b5a640 1 -- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 msgr2=0x7f08f00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-24T18:33:45.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.558+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:33:45.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.558+0000 7f08f6b5a640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:33:45.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 10 monclient(hunting): handle_auth_done global_id 4108 payload 824 2026-02-24T18:33:45.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 -- 10.20.193.18:0/142147654 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 msgr2=0x7f08f00c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:45.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 0x7f08f00c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 -- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 msgr2=0x7f08f00c8350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 10 monclient: _finish_hunting 0 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 monclient: found mon.noname-b 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.559+0000 7f08f6359640 1 -- 10.20.193.18:0/142147654 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f00c6a90 con 0x7f08d000b2b0 2026-02-24T18:33:45.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.561+0000 7f08f6359640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 0x7f08f00c9e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08e0002410 tx=0x7f08e0009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:45.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.561+0000 7f08d6ffd640 10 client.?.objecter ms_handle_connect 0x7f08d000b2b0 2026-02-24T18:33:45.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.561+0000 7f08d6ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0017410 con 0x7f08d000b2b0 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: dump: 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:45.567 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e001c070 con 0x7f08d000b2b0 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0017810 con 0x7f08d000b2b0 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:45.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08d6ffd640 10 monclient: dump: 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f4b56640 4 set_mon_vals no callback set 2026-02-24T18:33:45.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 5 monclient: authenticate success, global_id 4108 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 10 monclient: _renew_subs 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f00c6d70 con 0x7f08d000b2b0 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 10 client.4108.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 10 monclient: _renew_subs 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f00c7330 con 0x7f08d000b2b0 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.564+0000 7f08f85e4640 1 librados: init done 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.565+0000 7f08f85e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.565+0000 7f08f85e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.565+0000 7f08f85e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.565+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f0151ed0 con 0x7f08d000b2b0 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f08e0029070 con 0x7f08d000b2b0 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f08e0021ac0 con 0x7f08d000b2b0 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 10 client.4108.objecter ms_dispatch2 0x7f08f0000b90 osd_map(1..1 src has 1..1) 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 3 client.4108.objecter handle_osd_map got epochs [1,1] > 0 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 3 client.4108.objecter handle_osd_map decoding full epoch 1 2026-02-24T18:33:45.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.575+0000 7f08d6ffd640 20 client.4108.objecter dump_active .. 0 homeless 2026-02-24T18:33:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.653+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f08f0151ed0 con 0x7f08d000b2b0 2026-02-24T18:33:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.653+0000 7f08d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.653+0000 7f08d6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:45.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.683+0000 7f08f85e4640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-24T18:33:45.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.683+0000 7f08f85e4640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-24T18:33:45.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.683+0000 7f08f85e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:45.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.683+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f08f0152bd0 con 0x7f08d000b2b0 2026-02-24T18:33:45.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08d6ffd640 1 -- 10.20.193.18:0/142147654 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f08e0026020 con 0x7f08d000b2b0 2026-02-24T18:33:45.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-24T18:33:45.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08d6ffd640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-24T18:33:45.834 INFO:teuthology.orchestra.run.trial018.stderr:adjusted tunables profile to default 2026-02-24T18:33:45.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 10 librados: watch_flush enter 2026-02-24T18:33:45.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 10 librados: watch_flush exit 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 20 client.4108.objecter shutdown clearing up homeless session... 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 10 client.4108.objecter successfully canceled tick 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 10 monclient: shutdown 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 msgr2=0x7f08f00c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.832+0000 7f08f85e4640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 0x7f08f00c9e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08e0002410 tx=0x7f08e0009a50 comp rx=0 tx=0).stop 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 shutdown_connections 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f08d8004900 0x7f08f00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f08d8003c90 0x7f08f00c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 --2- 10.20.193.18:0/142147654 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08d000b2b0 0x7f08f00c9e50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 >> 10.20.193.18:0/142147654 conn(0x7f08f00ce730 msgr2=0x7f08f006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 shutdown_connections 2026-02-24T18:33:45.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 -- 10.20.193.18:0/142147654 wait complete. 2026-02-24T18:33:45.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:45.833+0000 7f08f85e4640 1 librados: shutdown 2026-02-24T18:33:45.843 INFO:tasks.ceph:check_enable_crimson: False 2026-02-24T18:33:45.843 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-24T18:33:45.844 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:45.844 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-24T18:33:45.868 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:45.868 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-24T18:33:45.922 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:45.920+0000 7f82a2bdc140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-24T18:33:45.931 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:45.931 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-24T18:33:45.951 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:45.949+0000 7f9c4642d140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-24T18:33:45.973 DEBUG:teuthology.orchestra.run.trial018:> set -ex 2026-02-24T18:33:45.974 DEBUG:teuthology.orchestra.run.trial018:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2026-02-24T18:33:46.036 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:46.036 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-24T18:33:46.043 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.040+0000 7f82a2bdc140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-24T18:33:46.063 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:46.063 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-24T18:33:46.070 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.067+0000 7f82a2bdc140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-24T18:33:46.073 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.072+0000 7f9c4642d140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-24T18:33:46.101 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.100+0000 7f9c4642d140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-24T18:33:46.103 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.100+0000 7f82a2bdc140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-24T18:33:46.126 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.124+0000 7f82a2bdc140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-24T18:33:46.128 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:46.128 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-24T18:33:46.135 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.133+0000 7f9c4642d140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-24T18:33:46.153 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.150+0000 7f82a2bdc140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-24T18:33:46.158 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.157+0000 7f9c4642d140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-24T18:33:46.184 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.183+0000 7f9c4642d140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-24T18:33:46.190 DEBUG:teuthology.orchestra.run.trial032:> set -ex 2026-02-24T18:33:46.191 DEBUG:teuthology.orchestra.run.trial032:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2026-02-24T18:33:46.209 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.206+0000 7f82a2bdc140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-24T18:33:46.241 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.239+0000 7f9c4642d140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-24T18:33:46.255 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:46.255 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-24T18:33:46.270 INFO:tasks.ceph.mgr.x.trial112.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T18:33:46.270 INFO:tasks.ceph.mgr.x.trial112.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T18:33:46.270 INFO:tasks.ceph.mgr.x.trial112.stderr: from numpy import show_config as show_numpy_config 2026-02-24T18:33:46.273 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.270+0000 7f82a2bdc140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-24T18:33:46.282 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:46.282 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-24T18:33:46.299 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.296+0000 7f82a2bdc140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-24T18:33:46.302 INFO:tasks.ceph.mgr.y.trial018.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T18:33:46.302 INFO:tasks.ceph.mgr.y.trial018.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T18:33:46.302 INFO:tasks.ceph.mgr.y.trial018.stderr: from numpy import show_config as show_numpy_config 2026-02-24T18:33:46.306 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.304+0000 7f9c4642d140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-24T18:33:46.331 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.330+0000 7f9c4642d140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-24T18:33:46.347 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:46.347 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2026-02-24T18:33:46.354 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.351+0000 7f82a2bdc140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-24T18:33:46.387 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.385+0000 7f9c4642d140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-24T18:33:46.397 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.395+0000 7f82a2bdc140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-24T18:33:46.410 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T18:33:46.410 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2026-02-24T18:33:46.423 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.420+0000 7f82a2bdc140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-24T18:33:46.430 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.429+0000 7f9c4642d140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-24T18:33:46.447 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.445+0000 7f82a2bdc140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-24T18:33:46.456 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.455+0000 7f9c4642d140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-24T18:33:46.473 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.470+0000 7f82a2bdc140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-24T18:33:46.474 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:46.474 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-24T18:33:46.481 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.480+0000 7f9c4642d140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-24T18:33:46.496 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.493+0000 7f82a2bdc140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-24T18:33:46.497 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:46.498 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-24T18:33:46.507 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.505+0000 7f9c4642d140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-24T18:33:46.529 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.528+0000 7f9c4642d140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-24T18:33:46.563 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:46.563 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2026-02-24T18:33:46.629 DEBUG:teuthology.orchestra.run.trial112:> set -ex 2026-02-24T18:33:46.630 DEBUG:teuthology.orchestra.run.trial112:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2026-02-24T18:33:46.693 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.689+0000 7f82a2bdc140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-24T18:33:46.694 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 29f6c438-7191-4921-a473-a7c822283cee 0 2026-02-24T18:33:46.725 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.723+0000 7f9c4642d140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-24T18:33:46.747 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.744+0000 7f82a2bdc140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-24T18:33:46.770 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.767+0000 7f82a2bdc140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-24T18:33:46.780 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.778+0000 7f9c4642d140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-24T18:33:46.794 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.791+0000 7f82a2bdc140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-24T18:33:46.797 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.794+0000 7f722ce8f640 10 monclient: get_monmap_and_config 2026-02-24T18:33:46.797 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.794+0000 7f722ce8f640 10 monclient: build_initial_monmap 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.794+0000 7f722ce8f640 10 monclient: monmap: 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:46.795973+0000 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:46.795973+0000 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:46.798 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:46.799 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:46.799 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:46.799 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.801 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 Processor -- start 2026-02-24T18:33:46.801 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 -- start start 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient: init 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient: _add_conns ranks=[3,1,2,0,4,5] 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f722811a950 0x7f722811ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-a con 0x7f722811a950 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f72281151a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7228115d40 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7228123e20 con 0x7f72281151a0 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7228125040 con 0x7f7228115d40 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:46.802 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7228126240 con 0x7f722811a950 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f722ce8f640 10 monclient(hunting): authenticate will time out at 1368.441415s 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f7226575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f722811a950 0x7f722811ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.795+0000 7f7225d74640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 0x7f7228115630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52148/0 (socket says 10.20.193.112:52148) 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225d74640 1 -- 10.20.193.112:0/3293857584 learned_addr learned my addr 10.20.193.112:0/3293857584 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): get_auth_request con 0x7f722811a950 auth_method 0 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:46.803 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225d74640 1 -- 10.20.193.112:0/3293857584 >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 legacy=0x7f7228115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225d74640 1 -- 10.20.193.112:0/3293857584 >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 legacy=0x7f7228115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225d74640 1 --1- 10.20.193.112:0/3293857584 >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 0x7f7228115630 :-1 s=OPENED pgs=14 cs=1 l=1).handle_message read tag failed 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225d74640 1 --1- 10.20.193.112:0/3293857584 >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 0x7f7228115630 :-1 s=OPENED pgs=14 cs=1 l=1).fault on lossy channel, failing 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.796+0000 7f7225573640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.32:6789/0 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 10 monclient(hunting): handle_auth_done global_id 4110 payload 274 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 1 -- 10.20.193.112:0/3293857584 >> v1:10.20.193.110:6789/0 conn(0x7f7228115d40 legacy=0x7f7228122640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 1 -- 10.20.193.112:0/3293857584 >> v1:10.20.193.32:6789/0 conn(0x7f72281151a0 legacy=0x7f7228115630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 10 monclient: _finish_hunting 0 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 1 monclient: found mon.noname-a 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:46.804 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.805 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.797+0000 7f7226575640 1 -- 10.20.193.112:0/3293857584 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72281263c0 con 0x7f722811a950 2026-02-24T18:33:46.805 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.798+0000 7f7226575640 1 --2- 10.20.193.112:0/3293857584 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f722811a950 0x7f722811ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f722811b420 tx=0x7f721c02ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5309d68e0bf6fc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:46.805 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.801+0000 7f9c4642d140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-24T18:33:46.813 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 1 -- 10.20.193.112:0/3293857584 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c03c070 con 0x7f722811a950 2026-02-24T18:33:46.813 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 10 monclient: dump: 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:46.814 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 10 monclient: _finish_auth 0 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 10 monclient: _check_auth_tickets 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f7225573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f722ce8f640 5 monclient: authenticate success, global_id 4110 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.800+0000 7f722ce8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 1 -- 10.20.193.112:0/3293857584 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721c02fa40 con 0x7f722811a950 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 1 -- 10.20.193.112:0/3293857584 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c02fe10 con 0x7f722811a950 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:46.815 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f7225573640 10 monclient: dump: 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:46.816 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 4 set_mon_vals no callback set 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 10 monclient: shutdown 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 -- 10.20.193.112:0/3293857584 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f722811a950 msgr2=0x7f722811ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 --2- 10.20.193.112:0/3293857584 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f722811a950 0x7f722811ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f722811b420 tx=0x7f721c02ed30 comp rx=0 tx=0).stop 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 -- 10.20.193.112:0/3293857584 shutdown_connections 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 --2- 10.20.193.112:0/3293857584 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f722811a950 0x7f722811ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 -- 10.20.193.112:0/3293857584 >> 10.20.193.112:0/3293857584 conn(0x7f722806dfb0 msgr2=0x7f722806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 -- 10.20.193.112:0/3293857584 shutdown_connections 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 1 -- 10.20.193.112:0/3293857584 wait complete. 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 10 monclient: build_initial_monmap 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.801+0000 7f722ce8f640 10 monclient: monmap: 2026-02-24T18:33:46.817 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 librados: starting msgr at 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 librados: starting objecter 2026-02-24T18:33:46.818 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 Processor -- start 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 -- start start 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 librados: setting wanted keys 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 librados: calling monclient init 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient: init 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-a con 0x7f72281151a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7228115d40 0x7f7228122ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-b con 0x7f7228115d40 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f722811a950 0x7f72281b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): picked mon.noname-c con 0x7f722811a950 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:46.819 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f72281256d0 con 0x7f72281151a0 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7228125550 con 0x7f7228115d40 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7228125850 con 0x7f722811a950 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f722ce8f640 10 monclient(hunting): authenticate will time out at 1368.448377s 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226d76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f722811a950 0x7f72281b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7225d74640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7228115d40 0x7f7228122ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59720/0 (socket says 10.20.193.112:59720) 2026-02-24T18:33:46.820 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 1 -- 10.20.193.112:0/2004007199 learned_addr learned my addr 10.20.193.112:0/2004007199 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 10 monclient(hunting): get_auth_request con 0x7f72281151a0 auth_method 0 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7226575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7225d74640 10 monclient(hunting): get_auth_request con 0x7f7228115d40 auth_method 0 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7225d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7225d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.802+0000 7f7225d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): get_auth_request con 0x7f722811a950 auth_method 0 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:46.821 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7225d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7225d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7225d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.803+0000 7f7226d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 10 monclient(hunting): handle_auth_done global_id 4113 payload 824 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 -- 10.20.193.112:0/2004007199 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f722811a950 msgr2=0x7f72281b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f722811a950 0x7f72281b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 -- 10.20.193.112:0/2004007199 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7228115d40 msgr2=0x7f7228122ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:46.822 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7228115d40 0x7f7228122ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 10 monclient: _finish_hunting 0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 monclient: found mon.noname-a 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.804+0000 7f7226575640 1 -- 10.20.193.112:0/2004007199 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72281b1ba0 con 0x7f72281151a0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.805+0000 7f7226575640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f721c03a040 tx=0x7f721c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.805+0000 7f720effd640 10 client.?.objecter ms_handle_connect 0x7f72281151a0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.805+0000 7f720effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c03c040 con 0x7f72281151a0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:46.823 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: dump: 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: _finish_auth 0 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:46.824 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 5 monclient: authenticate success, global_id 4113 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721c047070 con 0x7f72281151a0 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c0358a0 con 0x7f72281151a0 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f720effd640 10 monclient: dump: 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:46.825 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f7224d72640 4 set_mon_vals no callback set 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 10 monclient: _renew_subs 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72281b1dd0 con 0x7f72281151a0 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 10 client.4113.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 10 monclient: _renew_subs 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.826 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281b8480 con 0x7f72281151a0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.807+0000 7f722ce8f640 1 librados: init done 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.808+0000 7f722ce8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.808+0000 7f722ce8f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.808+0000 7f722ce8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.808+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722811ad30 con 0x7f72281151a0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f721c035a40 con 0x7f72281151a0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f721c04ca90 con 0x7f72281151a0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 10 client.4113.objecter ms_dispatch2 0x7f7228000b90 osd_map(2..2 src has 1..2) 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 3 client.4113.objecter handle_osd_map got epochs [2,2] > 0 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 3 client.4113.objecter handle_osd_map decoding full epoch 2 2026-02-24T18:33:46.827 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.810+0000 7f720effd640 20 client.4113.objecter dump_active .. 0 homeless 2026-02-24T18:33:46.828 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.825+0000 7f9c4642d140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-24T18:33:46.847 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.844+0000 7f82a2bdc140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:33:46.859 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.856+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f721c035d60 con 0x7f72281151a0 2026-02-24T18:33:46.859 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.856+0000 7f720effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:46.859 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.856+0000 7f720effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:46.868 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.865+0000 7f82a2bdc140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-24T18:33:46.880 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.879+0000 7f9c4642d140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:33:46.891 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.888+0000 7f722ce8f640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "29f6c438-7191-4921-a473-a7c822283cee", "id": 0}] 2026-02-24T18:33:46.891 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.888+0000 7f722ce8f640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "29f6c438-7191-4921-a473-a7c822283cee", "id": 0}] 2026-02-24T18:33:46.891 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.888+0000 7f722ce8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:46.892 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.888+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "29f6c438-7191-4921-a473-a7c822283cee", "id": 0} v 0) -- 0x7f722811b710 con 0x7f72281151a0 2026-02-24T18:33:46.901 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.900+0000 7f9c4642d140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-24T18:33:46.920 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:46.917+0000 7f82a2bdc140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:33:46.935 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.932+0000 7f720effd640 1 -- 10.20.193.112:0/2004007199 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "29f6c438-7191-4921-a473-a7c822283cee", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f721c04cce0 con 0x7f72281151a0 2026-02-24T18:33:46.936 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.932+0000 7f720effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "29f6c438-7191-4921-a473-a7c822283cee", "id": 0}] 2026-02-24T18:33:46.936 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.932+0000 7f720effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:46.936 INFO:teuthology.orchestra.run.trial112.stdout:0 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 10 librados: watch_flush enter 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 10 librados: watch_flush exit 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 20 client.4113.objecter shutdown clearing up homeless session... 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 10 client.4113.objecter successfully canceled tick 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 10 monclient: shutdown 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 msgr2=0x7f7228122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f721c03a040 tx=0x7f721c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 shutdown_connections 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7228115d40 0x7f7228122ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.938 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f722811a950 0x7f72281b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 --2- 10.20.193.112:0/2004007199 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f72281151a0 0x7f7228122440 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:46.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 >> 10.20.193.112:0/2004007199 conn(0x7f722806dfb0 msgr2=0x7f7228118370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:46.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 shutdown_connections 2026-02-24T18:33:46.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 -- 10.20.193.112:0/2004007199 wait complete. 2026-02-24T18:33:46.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:46.933+0000 7f722ce8f640 1 librados: shutdown 2026-02-24T18:33:46.945 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 9df55bdb-d545-4f6a-a8c5-816acf7fbe1d 1 2026-02-24T18:33:46.955 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:46.953+0000 7f9c4642d140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: get_monmap_and_config 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: monmap: 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:47.047400+0000 2026-02-24T18:33:47.049 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:47.047400+0000 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:47.050 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.053 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 Processor -- start 2026-02-24T18:33:47.053 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 -- start start 2026-02-24T18:33:47.053 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: init 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient: _add_conns ranks=[3,0,5,2,1,4] 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff27011a950 0x7ff27011ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-a con 0x7ff27011a950 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2701151a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff270115d40 0x7ff270122640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-c con 0x7ff270115d40 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff270122c40 con 0x7ff2701151a0 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff270123e60 con 0x7ff27011a950 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff270123fe0 con 0x7ff270115d40 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.046+0000 7ff2787eb640 10 monclient(hunting): authenticate will time out at 1368.692803s 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276560640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff27011a950 0x7ff27011ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.054 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff270115d40 0x7ff270122640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff270115d40 0x7ff270122640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:51978/0 (socket says 10.20.193.112:51978) 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 1 -- 10.20.193.112:0/3561421584 learned_addr learned my addr 10.20.193.112:0/3561421584 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): get_auth_request con 0x7ff270115d40 auth_method 0 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276560640 10 monclient(hunting): get_auth_request con 0x7ff27011a950 auth_method 0 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276560640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276560640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.047+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.055 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 10 monclient(hunting): handle_auth_done global_id 4118 payload 274 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 1 -- 10.20.193.112:0/3561421584 >> v2:10.20.193.18:3300/0 conn(0x7ff27011a950 msgr2=0x7ff27011ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 1 --2- 10.20.193.112:0/3561421584 >> v2:10.20.193.18:3300/0 conn(0x7ff27011a950 0x7ff27011ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 1 -- 10.20.193.112:0/3561421584 >> v1:10.20.193.18:6789/0 conn(0x7ff2701151a0 legacy=0x7ff270115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 1 monclient: found mon.noname-c 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.048+0000 7ff276d61640 1 -- 10.20.193.112:0/3561421584 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff270124190 con 0x7ff270115d40 2026-02-24T18:33:47.056 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.049+0000 7ff276d61640 1 --2- 10.20.193.112:0/3561421584 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff270115d40 0x7ff270122640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff27011b420 tx=0x7ff26c02f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f6f0af968caf98 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.059 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 1 -- 10.20.193.112:0/3561421584 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c02fcf0 con 0x7ff270115d40 2026-02-24T18:33:47.059 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.059 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: dump: 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 1 -- 10.20.193.112:0/3561421584 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c0373f0 con 0x7ff270115d40 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 5 monclient: authenticate success, global_id 4118 2026-02-24T18:33:47.060 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 1 -- 10.20.193.112:0/3561421584 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0377c0 con 0x7ff270115d40 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 4 set_mon_vals no callback set 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff27555e640 10 monclient: dump: 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 10 monclient: shutdown 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 -- 10.20.193.112:0/3561421584 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff270115d40 msgr2=0x7ff270122640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.061 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 --2- 10.20.193.112:0/3561421584 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff270115d40 0x7ff270122640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff27011b420 tx=0x7ff26c02f230 comp rx=0 tx=0).stop 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 -- 10.20.193.112:0/3561421584 shutdown_connections 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 --2- 10.20.193.112:0/3561421584 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff270115d40 0x7ff270122640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 --2- 10.20.193.112:0/3561421584 >> v2:10.20.193.18:3300/0 conn(0x7ff27011a950 0x7ff27011ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 -- 10.20.193.112:0/3561421584 >> 10.20.193.112:0/3561421584 conn(0x7ff27006dfb0 msgr2=0x7ff27006e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 -- 10.20.193.112:0/3561421584 shutdown_connections 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.051+0000 7ff2787eb640 1 -- 10.20.193.112:0/3561421584 wait complete. 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient: monmap: 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 librados: starting msgr at 2026-02-24T18:33:47.062 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 librados: starting objecter 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 Processor -- start 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 -- start start 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 librados: setting wanted keys 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 librados: calling monclient init 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient: init 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-c con 0x7ff2701151a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff270115d40 0x7ff2701b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-a con 0x7ff270115d40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff27011a950 0x7ff2701ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): picked mon.noname-b con 0x7ff27011a950 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff270123270 con 0x7ff270115d40 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2701230f0 con 0x7ff27011a950 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2701233f0 con 0x7ff2701151a0 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.063 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff2787eb640 10 monclient(hunting): authenticate will time out at 1368.698482s 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff275d5f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff270115d40 0x7ff2701b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:51984/0 (socket says 10.20.193.112:51984) 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 1 -- 10.20.193.112:0/346664725 learned_addr learned my addr 10.20.193.112:0/346664725 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 10 monclient(hunting): get_auth_request con 0x7ff2701151a0 auth_method 0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.052+0000 7ff276560640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff27011a950 0x7ff2701ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): get_auth_request con 0x7ff270115d40 auth_method 0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): get_auth_request con 0x7ff27011a950 auth_method 0 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff275d5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.053+0000 7ff276d61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 10 monclient(hunting): handle_auth_done global_id 4121 payload 824 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 -- 10.20.193.112:0/346664725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff27011a950 msgr2=0x7ff2701ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff27011a950 0x7ff2701ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 -- 10.20.193.112:0/346664725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff270115d40 msgr2=0x7ff2701b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff270115d40 0x7ff2701b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 monclient: found mon.noname-c 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.054+0000 7ff276560640 1 -- 10.20.193.112:0/346664725 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2701abe50 con 0x7ff2701151a0 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.055+0000 7ff276560640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ebe0 tx=0x7ff26000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.055+0000 7ff256ffd640 10 client.?.objecter ms_handle_connect 0x7ff2701151a0 2026-02-24T18:33:47.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.055+0000 7ff256ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:47.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260017070 con 0x7ff2701151a0 2026-02-24T18:33:47.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: dump: 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26000cc40 con 0x7ff2701151a0 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 5 monclient: authenticate success, global_id 4121 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260013470 con 0x7ff2701151a0 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff256ffd640 10 monclient: dump: 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.067 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff274d5d640 4 set_mon_vals no callback set 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _renew_subs 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2701ac140 con 0x7ff2701151a0 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 client.4121.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _renew_subs 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2701b8490 con 0x7ff2701151a0 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 1 librados: init done 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.068 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.057+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27011ad30 con 0x7ff2701151a0 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff260024060 con 0x7ff2701151a0 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7ff260013c70 con 0x7ff2701151a0 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 10 client.4121.objecter ms_dispatch2 0x7ff270000b90 osd_map(3..3 src has 1..3) 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 3 client.4121.objecter handle_osd_map got epochs [3,3] > 0 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 3 client.4121.objecter handle_osd_map decoding full epoch 3 2026-02-24T18:33:47.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.081+0000 7ff256ffd640 20 client.4121.objecter dump_active .. 0 homeless 2026-02-24T18:33:47.137 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.134+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7ff260013610 con 0x7ff2701151a0 2026-02-24T18:33:47.137 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.134+0000 7ff256ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.137 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.134+0000 7ff256ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:47.169 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.166+0000 7ff2787eb640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9df55bdb-d545-4f6a-a8c5-816acf7fbe1d", "id": 1}] 2026-02-24T18:33:47.170 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.166+0000 7ff2787eb640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9df55bdb-d545-4f6a-a8c5-816acf7fbe1d", "id": 1}] 2026-02-24T18:33:47.170 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.166+0000 7ff2787eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.170 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.166+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9df55bdb-d545-4f6a-a8c5-816acf7fbe1d", "id": 1} v 0) -- 0x7ff27011c740 con 0x7ff2701151a0 2026-02-24T18:33:47.231 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff256ffd640 1 -- 10.20.193.112:0/346664725 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9df55bdb-d545-4f6a-a8c5-816acf7fbe1d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7ff260005a90 con 0x7ff2701151a0 2026-02-24T18:33:47.231 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff256ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9df55bdb-d545-4f6a-a8c5-816acf7fbe1d", "id": 1}] 2026-02-24T18:33:47.231 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff256ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:47.232 INFO:teuthology.orchestra.run.trial112.stdout:1 2026-02-24T18:33:47.234 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 10 librados: watch_flush enter 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 10 librados: watch_flush exit 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 20 client.4121.objecter shutdown clearing up homeless session... 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 10 client.4121.objecter successfully canceled tick 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 10 monclient: shutdown 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 msgr2=0x7ff2701b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ebe0 tx=0x7ff26000c520 comp rx=0 tx=0).stop 2026-02-24T18:33:47.235 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 shutdown_connections 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff27011a950 0x7ff2701ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff270115d40 0x7ff2701b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 --2- 10.20.193.112:0/346664725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff2701151a0 0x7ff2701b14a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 >> 10.20.193.112:0/346664725 conn(0x7ff27006dfb0 msgr2=0x7ff270114410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 shutdown_connections 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 -- 10.20.193.112:0/346664725 wait complete. 2026-02-24T18:33:47.236 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.227+0000 7ff2787eb640 1 librados: shutdown 2026-02-24T18:33:47.240 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 9361a34f-9434-4f10-9037-2c32be178beb 2 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: get_monmap_and_config 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: monmap: 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:47.303584+0000 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:47.303584+0000 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.306 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 Processor -- start 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 -- start start 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: init 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient: _add_conns ranks=[3,2,0,4,5,1] 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f40001128f0 0x7f4000112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): picked mon.noname-a con 0x7f40001128f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4000117210 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4000117d70 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4000121b60 con 0x7f4000117d70 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4000122d80 con 0x7f4000117210 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f4000123f80 con 0x7f40001128f0 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.307 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.302+0000 7f4006043640 10 monclient(hunting): authenticate will time out at 1368.948917s 2026-02-24T18:33:47.308 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3fff7fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f40001128f0 0x7f4000112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.308 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3ffeffd640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f4000117210 0x7f4000117660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.112:51142/0 (socket says 10.20.193.112:51142) 2026-02-24T18:33:47.308 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3ffeffd640 1 -- 10.20.193.112:0/1795106767 learned_addr learned my addr 10.20.193.112:0/1795106767 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3fff7fe640 10 monclient(hunting): get_auth_request con 0x7f40001128f0 auth_method 0 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3fff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3fff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.303+0000 7f3fff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 1 -- 10.20.193.112:0/1795106767 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421103956 0 0) 0x7f4000122d80 con 0x7f4000117210 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 10 monclient(hunting): my global_id is 4124 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.304+0000 7f3ffe7fc640 1 -- 10.20.193.112:0/1795106767 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fe4003620 con 0x7f4000117210 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_done global_id 4122 payload 274 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 1 -- 10.20.193.112:0/1795106767 >> v1:10.20.193.110:6789/0 conn(0x7f4000117210 legacy=0x7f4000117660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 1 -- 10.20.193.112:0/1795106767 >> v1:10.20.193.18:6789/0 conn(0x7f4000117d70 legacy=0x7f40001203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 1 monclient: found mon.noname-a 2026-02-24T18:33:47.310 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.310 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:47.310 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.305+0000 7f3fff7fe640 1 -- 10.20.193.112:0/1795106767 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4000124100 con 0x7f40001128f0 2026-02-24T18:33:47.312 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.309+0000 7f3fff7fe640 1 --2- 10.20.193.112:0/1795106767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000112cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098a0 tx=0x7f3fec02edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7f3a69cdaef4c0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 1 -- 10.20.193.112:0/1795106767 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec03c070 con 0x7f40001128f0 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: dump: 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.317 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f4006043640 5 monclient: authenticate success, global_id 4122 2026-02-24T18:33:47.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 1 -- 10.20.193.112:0/1795106767 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec02fae0 con 0x7f40001128f0 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f4006043640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f4006043640 4 set_mon_vals no callback set 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 1 -- 10.20.193.112:0/1795106767 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec037440 con 0x7f40001128f0 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f3ffe7fc640 10 monclient: dump: 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.319 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.311+0000 7f4006043640 10 monclient: shutdown 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- 10.20.193.112:0/1795106767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 msgr2=0x7f4000112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 --2- 10.20.193.112:0/1795106767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000112cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098a0 tx=0x7f3fec02edd0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- 10.20.193.112:0/1795106767 shutdown_connections 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 --2- 10.20.193.112:0/1795106767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000112cd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- 10.20.193.112:0/1795106767 >> 10.20.193.112:0/1795106767 conn(0x7f4000071570 msgr2=0x7f4000073990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- 10.20.193.112:0/1795106767 shutdown_connections 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- 10.20.193.112:0/1795106767 wait complete. 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient: monmap: 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.321 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 librados: starting msgr at 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 librados: starting objecter 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 Processor -- start 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- start start 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 librados: setting wanted keys 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 librados: calling monclient init 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient: init 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.322 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000120080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): picked mon.noname-a con 0x7f40001128f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 0x7f40001205c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): picked mon.noname-c con 0x7f4000117210 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 0x7f40001b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): picked mon.noname-b con 0x7f4000117d70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f40001221f0 con 0x7f40001128f0 2026-02-24T18:33:47.323 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.324 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:47.312+0000 7f82a2bdc140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4000122070 con 0x7f4000117d70 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4000122370 con 0x7f4000117210 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.312+0000 7f4006043640 10 monclient(hunting): authenticate will time out at 1368.958654s 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 0x7f40001205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 0x7f40001b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 0x7f40001205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:51986/0 (socket says 10.20.193.112:51986) 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 1 -- 10.20.193.112:0/404814532 learned_addr learned my addr 10.20.193.112:0/404814532 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): get_auth_request con 0x7f4000117210 auth_method 0 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): get_auth_request con 0x7f4000117d70 auth_method 0 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3ffeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.313+0000 7f3fff7fe640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000120080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): get_auth_request con 0x7f40001128f0 auth_method 0 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3fff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 -- 10.20.193.112:0/404814532 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 msgr2=0x7f40001205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 0x7f40001205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 -- 10.20.193.112:0/404814532 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 msgr2=0x7f4000120080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000120080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 monclient: found mon.noname-b 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.314+0000 7f3ffffff640 1 -- 10.20.193.112:0/404814532 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001b7b80 con 0x7f4000117d70 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.315+0000 7f3ffffff640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 0x7f40001b7540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400cb40 tx=0x7f3ff400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.315+0000 7f3fdbfff640 10 client.?.objecter ms_handle_connect 0x7f4000117d70 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.315+0000 7f3fdbfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400ad50 con 0x7f4000117d70 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: dump: 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.333 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff4003a30 con 0x7f4000117d70 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4003e00 con 0x7f4000117d70 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3fdbfff640 10 monclient: dump: 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.334 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f3ffdffb640 4 set_mon_vals no callback set 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 5 monclient: authenticate success, global_id 4120 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 10 monclient: _renew_subs 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001b7e10 con 0x7f4000117d70 2026-02-24T18:33:47.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 10 client.4120.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 10 monclient: _renew_subs 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001b82f0 con 0x7f4000117d70 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.317+0000 7f4006043640 1 librados: init done 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.318+0000 7f4006043640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.318+0000 7f4006043640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.318+0000 7f4006043640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.318+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000112d50 con 0x7f4000117d70 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3ff40050d0 con 0x7f4000117d70 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3ff4025d00 con 0x7f4000117d70 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 10 client.4120.objecter ms_dispatch2 0x7f4000000b90 osd_map(4..4 src has 1..4) 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 3 client.4120.objecter handle_osd_map got epochs [4,4] > 0 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 3 client.4120.objecter handle_osd_map decoding full epoch 4 2026-02-24T18:33:47.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.319+0000 7f3fdbfff640 20 client.4120.objecter dump_active .. 0 homeless 2026-02-24T18:33:47.338 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:47.335+0000 7f82a2bdc140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-24T18:33:47.356 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:47.354+0000 7f9c4642d140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-24T18:33:47.360 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.357+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f3ff4025430 con 0x7f4000117d70 2026-02-24T18:33:47.360 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.357+0000 7f3fdbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.360 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.357+0000 7f3fdbfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:47.378 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:47.377+0000 7f9c4642d140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-24T18:33:47.392 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.389+0000 7f4006043640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9361a34f-9434-4f10-9037-2c32be178beb", "id": 2}] 2026-02-24T18:33:47.392 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.389+0000 7f4006043640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9361a34f-9434-4f10-9037-2c32be178beb", "id": 2}] 2026-02-24T18:33:47.392 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.389+0000 7f4006043640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.392 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.389+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9361a34f-9434-4f10-9037-2c32be178beb", "id": 2} v 0) -- 0x7f40001193a0 con 0x7f4000117d70 2026-02-24T18:33:47.455 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.452+0000 7f3fdbfff640 1 -- 10.20.193.112:0/404814532 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9361a34f-9434-4f10-9037-2c32be178beb", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3ff40258e0 con 0x7f4000117d70 2026-02-24T18:33:47.455 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.452+0000 7f3fdbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9361a34f-9434-4f10-9037-2c32be178beb", "id": 2}] 2026-02-24T18:33:47.455 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.452+0000 7f3fdbfff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:47.455 INFO:teuthology.orchestra.run.trial112.stdout:2 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 10 librados: watch_flush enter 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 10 librados: watch_flush exit 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 20 client.4120.objecter shutdown clearing up homeless session... 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 10 client.4120.objecter successfully canceled tick 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 10 monclient: shutdown 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 msgr2=0x7f40001b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 0x7f40001b7540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400cb40 tx=0x7f3ff400a430 comp rx=0 tx=0).stop 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 shutdown_connections 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4000117d70 0x7f40001b7540 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4000117210 0x7f40001205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 --2- 10.20.193.112:0/404814532 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f40001128f0 0x7f4000120080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 >> 10.20.193.112:0/404814532 conn(0x7f4000071570 msgr2=0x7f4000073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 shutdown_connections 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 -- 10.20.193.112:0/404814532 wait complete. 2026-02-24T18:33:47.456 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.453+0000 7f4006043640 1 librados: shutdown 2026-02-24T18:33:47.463 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 733635aa-8ae4-43d7-a53a-f6a66a520fe8 3 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.523+0000 7f910d085640 10 monclient: get_monmap_and_config 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.523+0000 7f910d085640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.523+0000 7f910d085640 10 monclient: monmap: 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.526 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:47.524938+0000 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:47.524938+0000 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:47.527 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.528 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 Processor -- start 2026-02-24T18:33:47.528 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 -- start start 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient: init 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient: _add_conns ranks=[4,5,0,1,2,3] 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9108073930 0x7f9108073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): picked mon.noname-b con 0x7f9108073930 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9108074280 0x7f9108116200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): picked mon.noname-c con 0x7f9108074280 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91081168e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9108116f70 con 0x7f91081168e0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9108117990 con 0x7f9108073930 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9108117b10 con 0x7f9108074280 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f910d085640 10 monclient(hunting): authenticate will time out at 1369.170249s 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106d76640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9108073930 0x7f9108073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106575640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9108074280 0x7f9108116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9107577640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f91081168e0 0x7f9108117280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.112:52866/0 (socket says 10.20.193.112:52866) 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9107577640 1 -- 10.20.193.112:0/1575297517 learned_addr learned my addr 10.20.193.112:0/1575297517 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106d76640 10 monclient(hunting): get_auth_request con 0x7f9108073930 auth_method 0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106575640 10 monclient(hunting): get_auth_request con 0x7f9108074280 auth_method 0 2026-02-24T18:33:47.529 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.524+0000 7f9106575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient(hunting): handle_auth_done global_id 4130 payload 274 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 1 -- 10.20.193.112:0/1575297517 >> v2:10.20.193.32:3300/0 conn(0x7f9108073930 msgr2=0x7f9108073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 1 --2- 10.20.193.112:0/1575297517 >> v2:10.20.193.32:3300/0 conn(0x7f9108073930 0x7f9108073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 1 -- 10.20.193.112:0/1575297517 >> v1:10.20.193.18:6789/0 conn(0x7f91081168e0 legacy=0x7f9108117280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 1 monclient: found mon.noname-c 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.525+0000 7f9106575640 1 -- 10.20.193.112:0/1575297517 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9108117cc0 con 0x7f9108074280 2026-02-24T18:33:47.530 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.526+0000 7f9106575640 1 --2- 10.20.193.112:0/1575297517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f9108116200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9108074250 tx=0x7f90f002ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfd352b736a143e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.531 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 1 -- 10.20.193.112:0/1575297517 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f003c070 con 0x7f9108074280 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: dump: 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 5 monclient: authenticate success, global_id 4130 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 1 -- 10.20.193.112:0/1575297517 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f002fa60 con 0x7f9108074280 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 1 -- 10.20.193.112:0/1575297517 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f002fe30 con 0x7f9108074280 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f9105d74640 10 monclient: dump: 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.532 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 4 set_mon_vals no callback set 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 10 monclient: shutdown 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 -- 10.20.193.112:0/1575297517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 msgr2=0x7f9108116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 --2- 10.20.193.112:0/1575297517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f9108116200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9108074250 tx=0x7f90f002ed50 comp rx=0 tx=0).stop 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 -- 10.20.193.112:0/1575297517 shutdown_connections 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 --2- 10.20.193.112:0/1575297517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f9108116200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 --2- 10.20.193.112:0/1575297517 >> v2:10.20.193.32:3300/0 conn(0x7f9108073930 0x7f9108073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 -- 10.20.193.112:0/1575297517 >> 10.20.193.112:0/1575297517 conn(0x7f910806f640 msgr2=0x7f9108071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.528+0000 7f910d085640 1 -- 10.20.193.112:0/1575297517 shutdown_connections 2026-02-24T18:33:47.533 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 -- 10.20.193.112:0/1575297517 wait complete. 2026-02-24T18:33:47.535 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.535 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient: monmap: 2026-02-24T18:33:47.535 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.535 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:47.535 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 librados: starting msgr at 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 librados: starting objecter 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 Processor -- start 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 -- start start 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 librados: setting wanted keys 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 librados: calling monclient init 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient: init 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 0x7f91081afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): picked mon.noname-b con 0x7f9108073930 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): picked mon.noname-c con 0x7f9108074280 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91081168e0 0x7f91081b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): picked mon.noname-a con 0x7f91081168e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9108118350 con 0x7f91081168e0 2026-02-24T18:33:47.536 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91081181d0 con 0x7f9108073930 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91081184d0 con 0x7f9108074280 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.529+0000 7f910d085640 10 monclient(hunting): authenticate will time out at 1369.175730s 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 0x7f91081afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91081168e0 0x7f91081b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:52012/0 (socket says 10.20.193.112:52012) 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 0x7f91081afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38628/0 (socket says 10.20.193.112:38628) 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 1 -- 10.20.193.112:0/2807504862 learned_addr learned my addr 10.20.193.112:0/2807504862 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): get_auth_request con 0x7f9108074280 auth_method 0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): get_auth_request con 0x7f9108073930 auth_method 0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): get_auth_request con 0x7f91081168e0 auth_method 0 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.537 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9106d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.530+0000 7f9107577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 10 monclient(hunting): handle_auth_done global_id 4133 payload 824 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 -- 10.20.193.112:0/2807504862 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 msgr2=0x7f91081afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 0x7f91081afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 -- 10.20.193.112:0/2807504862 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91081168e0 msgr2=0x7f91081b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91081168e0 0x7f91081b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 monclient: found mon.noname-c 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.531+0000 7f9106575640 1 -- 10.20.193.112:0/2807504862 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91081b4e00 con 0x7f9108074280 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.532+0000 7f9106575640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009130 tx=0x7f90f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.532+0000 7f90e77fe640 10 client.?.objecter ms_handle_connect 0x7f9108074280 2026-02-24T18:33:47.538 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.532+0000 7f90e77fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:47.539 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f003c040 con 0x7f9108074280 2026-02-24T18:33:47.539 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.539 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:47.539 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: dump: 2026-02-24T18:33:47.539 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0042450 con 0x7f9108074280 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 5 monclient: authenticate success, global_id 4133 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0042820 con 0x7f9108074280 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f90e77fe640 10 monclient: dump: 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.540 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f9105573640 4 set_mon_vals no callback set 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _renew_subs 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081b5090 con 0x7f9108074280 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 client.4133.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _renew_subs 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081bc7a0 con 0x7f9108074280 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 1 librados: init done 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.534+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9108073d10 con 0x7f9108074280 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f90f0055430 con 0x7f9108074280 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f90f004b8a0 con 0x7f9108074280 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 10 client.4133.objecter ms_dispatch2 0x7f9108000b90 osd_map(5..5 src has 1..5) 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 3 client.4133.objecter handle_osd_map got epochs [5,5] > 0 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 3 client.4133.objecter handle_osd_map decoding full epoch 5 2026-02-24T18:33:47.541 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.536+0000 7f90e77fe640 20 client.4133.objecter dump_active .. 0 homeless 2026-02-24T18:33:47.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.576+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f90f004b410 con 0x7f9108074280 2026-02-24T18:33:47.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.576+0000 7f90e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.576+0000 7f90e77fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:47.611 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.608+0000 7f910d085640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "733635aa-8ae4-43d7-a53a-f6a66a520fe8", "id": 3}] 2026-02-24T18:33:47.611 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.608+0000 7f910d085640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "733635aa-8ae4-43d7-a53a-f6a66a520fe8", "id": 3}] 2026-02-24T18:33:47.611 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.608+0000 7f910d085640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:47.611 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.608+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd new", "uuid": "733635aa-8ae4-43d7-a53a-f6a66a520fe8", "id": 3} v 0) -- 0x7f91081a9100 con 0x7f9108074280 2026-02-24T18:33:47.645 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f90e77fe640 1 -- 10.20.193.112:0/2807504862 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "733635aa-8ae4-43d7-a53a-f6a66a520fe8", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f90f004b6c0 con 0x7f9108074280 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f90e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "733635aa-8ae4-43d7-a53a-f6a66a520fe8", "id": 3}] 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f90e77fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stdout:3 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 10 librados: watch_flush enter 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 10 librados: watch_flush exit 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 20 client.4133.objecter shutdown clearing up homeless session... 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 10 client.4133.objecter successfully canceled tick 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 10 monclient: shutdown 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 msgr2=0x7f91081b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009130 tx=0x7f90f00047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 shutdown_connections 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91081168e0 0x7f91081b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9108074280 0x7f91081b0380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 --2- 10.20.193.112:0/2807504862 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9108073930 0x7f91081afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 >> 10.20.193.112:0/2807504862 conn(0x7f910806f640 msgr2=0x7f9108114a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.646 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 shutdown_connections 2026-02-24T18:33:47.647 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 -- 10.20.193.112:0/2807504862 wait complete. 2026-02-24T18:33:47.647 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.643+0000 7f910d085640 1 librados: shutdown 2026-02-24T18:33:47.653 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 56308d78-75f5-4853-ac45-b958a4dc7cce 4 2026-02-24T18:33:47.682 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:47.680+0000 7f82a2bdc140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: get_monmap_and_config 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: monmap: 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:47.715653+0000 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:47.715653+0000 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.717 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 Processor -- start 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 -- start start 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: init 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient: _add_conns ranks=[3,4,2,1,0,5] 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3a10073930 0x7f3a10073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-a con 0x7f3a10073930 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f3a10074280 0x7f3a10116200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-b con 0x7f3a10074280 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3a101168e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a10116f70 con 0x7f3a101168e0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3a10117990 con 0x7f3a10073930 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3a10117b10 con 0x7f3a10074280 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.714+0000 7f3a15385640 10 monclient(hunting): authenticate will time out at 1369.360998s 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0f7fe640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f3a101168e0 0x7f3a10117280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.112:51156/0 (socket says 10.20.193.112:51156) 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0f7fe640 1 -- 10.20.193.112:0/1279834124 learned_addr learned my addr 10.20.193.112:0/1279834124 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 1 --2- 10.20.193.112:0/1279834124 >> v2:10.20.193.32:3300/0 conn(0x7f3a10074280 0x7f3a10116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): get_auth_request con 0x7f3a10074280 auth_method 0 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.718 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.715+0000 7f3a0e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 10 monclient(hunting): handle_auth_done global_id 4129 payload 274 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 1 -- 10.20.193.112:0/1279834124 >> v2:10.20.193.18:3300/0 conn(0x7f3a10073930 msgr2=0x7f3a10073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 1 --2- 10.20.193.112:0/1279834124 >> v2:10.20.193.18:3300/0 conn(0x7f3a10073930 0x7f3a10073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 1 -- 10.20.193.112:0/1279834124 >> v1:10.20.193.110:6789/0 conn(0x7f3a101168e0 legacy=0x7f3a10117280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0dffb640 1 -- 10.20.193.112:0/1279834124 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528688525 0 0) 0x7f3a10116f70 con 0x7f3a101168e0 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 1 monclient: found mon.noname-b 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0e7fc640 1 -- 10.20.193.112:0/1279834124 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10117cc0 con 0x7f3a10074280 2026-02-24T18:33:47.719 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.716+0000 7f3a0dffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-24T18:33:47.720 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.717+0000 7f3a0e7fc640 1 --2- 10.20.193.112:0/1279834124 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10074280 0x7f3a10116200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a10074250 tx=0x7f39f802ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72d027e7b4c3d26 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.726 INFO:tasks.ceph.mgr.y.trial018.stderr:2026-02-24T18:33:47.725+0000 7f9c4642d140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 1 -- 10.20.193.112:0/1279834124 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f803c070 con 0x7f3a10074280 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: dump: 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 1 -- 10.20.193.112:0/1279834124 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f802fa60 con 0x7f3a10074280 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 5 monclient: authenticate success, global_id 4129 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 1 -- 10.20.193.112:0/1279834124 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f802fe30 con 0x7f3a10074280 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 4 set_mon_vals no callback set 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a0dffb640 10 monclient: dump: 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.728 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.729 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.729 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.729 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.729 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.729 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 10 monclient: shutdown 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 -- 10.20.193.112:0/1279834124 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10074280 msgr2=0x7f3a10116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 --2- 10.20.193.112:0/1279834124 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10074280 0x7f3a10116200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a10074250 tx=0x7f39f802ed50 comp rx=0 tx=0).stop 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 -- 10.20.193.112:0/1279834124 shutdown_connections 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 --2- 10.20.193.112:0/1279834124 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10074280 0x7f3a10116200 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 --2- 10.20.193.112:0/1279834124 >> v2:10.20.193.18:3300/0 conn(0x7f3a10073930 0x7f3a10073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 -- 10.20.193.112:0/1279834124 >> 10.20.193.112:0/1279834124 conn(0x7f3a1006f640 msgr2=0x7f3a10071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 -- 10.20.193.112:0/1279834124 shutdown_connections 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.725+0000 7f3a15385640 1 -- 10.20.193.112:0/1279834124 wait complete. 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient: build_initial_monmap 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient: monmap: 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 librados: starting msgr at 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 librados: starting objecter 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 Processor -- start 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 -- start start 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 librados: setting wanted keys 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 librados: calling monclient init 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient: init 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-b con 0x7f3a10073930 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a10074280 0x7f3a101b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-c con 0x7f3a10074280 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a101168e0 0x7f3a101b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): picked mon.noname-a con 0x7f3a101168e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10118350 con 0x7f3a101168e0 2026-02-24T18:33:47.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a101181d0 con 0x7f3a10073930 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a101184d0 con 0x7f3a10074280 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a15385640 10 monclient(hunting): authenticate will time out at 1369.372691s 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.726+0000 7f3a0effd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a101168e0 0x7f3a101b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38650/0 (socket says 10.20.193.112:38650) 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 1 -- 10.20.193.112:0/2335573531 learned_addr learned my addr 10.20.193.112:0/2335573531 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): get_auth_request con 0x7f3a10073930 auth_method 0 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): get_auth_request con 0x7f3a101168e0 auth_method 0 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.727+0000 7f3a0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 -- 10.20.193.112:0/2335573531 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a10074280 msgr2=0x7f3a101b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a10074280 0x7f3a101b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 -- 10.20.193.112:0/2335573531 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a101168e0 msgr2=0x7f3a101b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a101168e0 0x7f3a101b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 10 monclient: _finish_hunting 0 2026-02-24T18:33:47.731 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 monclient: found mon.noname-b 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.728+0000 7f3a0effd640 1 -- 10.20.193.112:0/2335573531 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a101b4cf0 con 0x7f3a10073930 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.729+0000 7f3a0effd640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b780 tx=0x7f3a0400bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.729+0000 7f39ef7fe640 10 client.?.objecter ms_handle_connect 0x7f3a10073930 2026-02-24T18:33:47.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.729+0000 7f39ef7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04004480 con 0x7f3a10073930 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: dump: 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.734 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: _finish_auth 0 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a04010070 con 0x7f3a10073930 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 5 monclient: authenticate success, global_id 4132 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _renew_subs 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a101b4fe0 con 0x7f3a10073930 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04002920 con 0x7f3a10073930 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f39ef7fe640 10 monclient: dump: 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a0d7fa640 4 set_mon_vals no callback set 2026-02-24T18:33:47.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 client.4132.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _renew_subs 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a101bc900 con 0x7f3a10073930 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 1 librados: init done 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.731+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10073d10 con 0x7f3a10073930 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.732+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3a0401d030 con 0x7f3a10073930 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.733+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3a04015980 con 0x7f3a10073930 2026-02-24T18:33:47.736 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.733+0000 7f39ef7fe640 10 client.4132.objecter ms_dispatch2 0x7f3a10000b90 osd_map(6..6 src has 1..6) 2026-02-24T18:33:47.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.733+0000 7f39ef7fe640 3 client.4132.objecter handle_osd_map got epochs [6,6] > 0 2026-02-24T18:33:47.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.733+0000 7f39ef7fe640 3 client.4132.objecter handle_osd_map decoding full epoch 6 2026-02-24T18:33:47.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.733+0000 7f39ef7fe640 20 client.4132.objecter dump_active .. 0 homeless 2026-02-24T18:33:47.774 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.771+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f3a04024080 con 0x7f3a10073930 2026-02-24T18:33:47.774 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.771+0000 7f39ef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:47.774 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.771+0000 7f39ef7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:47.806 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.803+0000 7f3a15385640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "56308d78-75f5-4853-ac45-b958a4dc7cce", "id": 4}] 2026-02-24T18:33:47.806 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.803+0000 7f3a15385640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "56308d78-75f5-4853-ac45-b958a4dc7cce", "id": 4}] 2026-02-24T18:33:47.806 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.803+0000 7f3a15385640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:47.806 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.803+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "56308d78-75f5-4853-ac45-b958a4dc7cce", "id": 4} v 0) -- 0x7f3a101a9100 con 0x7f3a10073930 2026-02-24T18:33:47.828 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.826+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f3a04015bd0 con 0x7f3a10073930 2026-02-24T18:33:47.986 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f39ef7fe640 1 -- 10.20.193.112:0/2335573531 <== mon.1 v2:10.20.193.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56308d78-75f5-4853-ac45-b958a4dc7cce", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3a04051240 con 0x7f3a10073930 2026-02-24T18:33:47.986 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f39ef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "56308d78-75f5-4853-ac45-b958a4dc7cce", "id": 4}] 2026-02-24T18:33:47.986 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f39ef7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:47.987 INFO:teuthology.orchestra.run.trial112.stdout:4 2026-02-24T18:33:47.989 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 10 librados: watch_flush enter 2026-02-24T18:33:47.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 10 librados: watch_flush exit 2026-02-24T18:33:47.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 20 client.4132.objecter shutdown clearing up homeless session... 2026-02-24T18:33:47.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 10 client.4132.objecter successfully canceled tick 2026-02-24T18:33:47.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 10 monclient: shutdown 2026-02-24T18:33:47.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 msgr2=0x7f3a101afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.982+0000 7f3a15385640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b780 tx=0x7f3a0400bc50 comp rx=0 tx=0).stop 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 shutdown_connections 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a101168e0 0x7f3a101b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a10074280 0x7f3a101b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 --2- 10.20.193.112:0/2335573531 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a10073930 0x7f3a101afdc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:47.992 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 >> 10.20.193.112:0/2335573531 conn(0x7f3a1006f640 msgr2=0x7f3a10072820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:47.992 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 shutdown_connections 2026-02-24T18:33:47.992 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 -- 10.20.193.112:0/2335573531 wait complete. 2026-02-24T18:33:47.992 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:47.983+0000 7f3a15385640 1 librados: shutdown 2026-02-24T18:33:47.994 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 9d335fc6-9910-49e7-b3b4-2a66c119eebb 5 2026-02-24T18:33:48.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: get_monmap_and_config 2026-02-24T18:33:48.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: monmap: 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:48.095725+0000 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:48.095725+0000 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.100 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 1 Processor -- start 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 1 -- start start 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: init 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.094+0000 7fe561fe4640 10 monclient: _add_conns ranks=[2,5,3,1,4,0] 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe55c073930 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe55c117280 0x7fe55c117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-c con 0x7fe55c117280 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe55c117c50 0x7fe55c118100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-a con 0x7fe55c117c50 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55c127360 con 0x7fe55c073930 2026-02-24T18:33:48.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe55c118780 con 0x7fe55c117c50 2026-02-24T18:33:48.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe55c128540 con 0x7fe55c117280 2026-02-24T18:33:48.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.095+0000 7fe561fe4640 10 monclient(hunting): authenticate will time out at 1369.741092s 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe55c117c50 0x7fe55c118100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe55c117c50 0x7fe55c118100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59858/0 (socket says 10.20.193.112:59858) 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 1 -- 10.20.193.112:0/436058019 learned_addr learned my addr 10.20.193.112:0/436058019 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 10 monclient(hunting): get_auth_request con 0x7fe55c117c50 auth_method 0 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.108 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.105+0000 7fe55bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.109 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.106+0000 7fe55bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.109 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.106+0000 7fe55bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.109 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.106+0000 7fe55bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 10 monclient(hunting): handle_auth_done global_id 4149 payload 274 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 1 -- 10.20.193.112:0/436058019 >> v2:10.20.193.110:3300/0 conn(0x7fe55c117280 msgr2=0x7fe55c117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 1 --2- 10.20.193.112:0/436058019 >> v2:10.20.193.110:3300/0 conn(0x7fe55c117280 0x7fe55c117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 1 -- 10.20.193.112:0/436058019 >> v1:10.20.193.110:6789/0 conn(0x7fe55c073930 legacy=0x7fe55c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 1 monclient: found mon.noname-a 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.107+0000 7fe55bfff640 1 -- 10.20.193.112:0/436058019 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c1286f0 con 0x7fe55c117c50 2026-02-24T18:33:48.110 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.108+0000 7fe55bfff640 1 --2- 10.20.193.112:0/436058019 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c117c50 0x7fe55c118100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe55c074420 tx=0x7fe55002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebf9af5afb17df6 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.111 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.108+0000 7fe55a7fc640 1 -- 10.20.193.112:0/436058019 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55002fc50 con 0x7fe55c117c50 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 10 monclient: dump: 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe55a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe561fe4640 5 monclient: authenticate success, global_id 4149 2026-02-24T18:33:48.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.109+0000 7fe561fe4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 1 -- 10.20.193.112:0/436058019 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55002fdf0 con 0x7fe55c117c50 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 1 -- 10.20.193.112:0/436058019 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550037610 con 0x7fe55c117c50 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe55a7fc640 10 monclient: dump: 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.114 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 4 set_mon_vals no callback set 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 10 monclient: shutdown 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- 10.20.193.112:0/436058019 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c117c50 msgr2=0x7fe55c118100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 --2- 10.20.193.112:0/436058019 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c117c50 0x7fe55c118100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe55c074420 tx=0x7fe55002f230 comp rx=0 tx=0).stop 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- 10.20.193.112:0/436058019 shutdown_connections 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 --2- 10.20.193.112:0/436058019 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c117c50 0x7fe55c118100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 --2- 10.20.193.112:0/436058019 >> v2:10.20.193.110:3300/0 conn(0x7fe55c117280 0x7fe55c117710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- 10.20.193.112:0/436058019 >> 10.20.193.112:0/436058019 conn(0x7fe55c06f640 msgr2=0x7fe55c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- 10.20.193.112:0/436058019 shutdown_connections 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- 10.20.193.112:0/436058019 wait complete. 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 10 monclient: monmap: 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 librados: starting msgr at 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 librados: starting objecter 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 Processor -- start 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 -- start start 2026-02-24T18:33:48.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 librados: setting wanted keys 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 1 librados: calling monclient init 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.110+0000 7fe561fe4640 10 monclient: init 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-a con 0x7fe55c073930 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe55c117280 0x7fe55c1b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-b con 0x7fe55c117280 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe55c117c50 0x7fe55c1b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): picked mon.noname-c con 0x7fe55c117c50 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c127990 con 0x7fe55c073930 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c127810 con 0x7fe55c117280 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c127b10 con 0x7fe55c117c50 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe561fe4640 10 monclient(hunting): authenticate will time out at 1369.757106s 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59864/0 (socket says 10.20.193.112:59864) 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 1 -- 10.20.193.112:0/2500092541 learned_addr learned my addr 10.20.193.112:0/2500092541 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): get_auth_request con 0x7fe55c073930 auth_method 0 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.111+0000 7fe55b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.112+0000 7fe55affd640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe55c117280 0x7fe55c1b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 10 monclient(hunting): handle_auth_done global_id 4152 payload 824 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 -- 10.20.193.112:0/2500092541 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe55c117c50 msgr2=0x7fe55c1b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe55c117c50 0x7fe55c1b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 -- 10.20.193.112:0/2500092541 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe55c117280 msgr2=0x7fe55c1b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe55c117280 0x7fe55c1b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 monclient: found mon.noname-a 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.113+0000 7fe55b7fe640 1 -- 10.20.193.112:0/2500092541 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c1b4d10 con 0x7fe55c073930 2026-02-24T18:33:48.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.118+0000 7fe55b7fe640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe54800b570 tx=0x7fe54800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.118+0000 7fe537fff640 10 client.?.objecter ms_handle_connect 0x7fe55c073930 2026-02-24T18:33:48.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.118+0000 7fe537fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548013020 con 0x7fe55c073930 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: dump: 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 5 monclient: authenticate success, global_id 4152 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5480044a0 con 0x7fe55c073930 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _renew_subs 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c1b5000 con 0x7fe55c073930 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480029c0 con 0x7fe55c073930 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe559ffb640 4 set_mon_vals no callback set 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe537fff640 10 monclient: dump: 2026-02-24T18:33:48.128 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 client.4152.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _renew_subs 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1bc920 con 0x7fe55c073930 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 1 librados: init done 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.129 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.125+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c117790 con 0x7fe55c073930 2026-02-24T18:33:48.134 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.131+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7fe548002b60 con 0x7fe55c073930 2026-02-24T18:33:48.135 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.133+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe54800fda0 con 0x7fe55c073930 2026-02-24T18:33:48.135 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.133+0000 7fe537fff640 10 client.4152.objecter ms_dispatch2 0x7fe55c000b90 osd_map(7..7 src has 1..7) 2026-02-24T18:33:48.135 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.133+0000 7fe537fff640 3 client.4152.objecter handle_osd_map got epochs [7,7] > 0 2026-02-24T18:33:48.135 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.133+0000 7fe537fff640 3 client.4152.objecter handle_osd_map decoding full epoch 7 2026-02-24T18:33:48.135 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.133+0000 7fe537fff640 20 client.4152.objecter dump_active .. 0 homeless 2026-02-24T18:33:48.197 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.194+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fe54801d020 con 0x7fe55c073930 2026-02-24T18:33:48.197 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.194+0000 7fe537fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.197 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.194+0000 7fe537fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:48.229 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.226+0000 7fe561fe4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9d335fc6-9910-49e7-b3b4-2a66c119eebb", "id": 5}] 2026-02-24T18:33:48.229 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.226+0000 7fe561fe4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9d335fc6-9910-49e7-b3b4-2a66c119eebb", "id": 5}] 2026-02-24T18:33:48.229 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.226+0000 7fe561fe4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.229 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.226+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9d335fc6-9910-49e7-b3b4-2a66c119eebb", "id": 5} v 0) -- 0x7fe55c1bcaf0 con 0x7fe55c073930 2026-02-24T18:33:48.247 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe537fff640 1 -- 10.20.193.112:0/2500092541 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d335fc6-9910-49e7-b3b4-2a66c119eebb", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe54800f640 con 0x7fe55c073930 2026-02-24T18:33:48.247 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe537fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9d335fc6-9910-49e7-b3b4-2a66c119eebb", "id": 5}] 2026-02-24T18:33:48.247 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe537fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:48.247 INFO:teuthology.orchestra.run.trial112.stdout:5 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 10 librados: watch_flush enter 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 10 librados: watch_flush exit 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 20 client.4152.objecter shutdown clearing up homeless session... 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 10 client.4152.objecter successfully canceled tick 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 10 monclient: shutdown 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 msgr2=0x7fe55c1afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe54800b570 tx=0x7fe54800ba40 comp rx=0 tx=0).stop 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 shutdown_connections 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe55c117280 0x7fe55c1b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe55c117c50 0x7fe55c1b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 --2- 10.20.193.112:0/2500092541 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe55c073930 0x7fe55c1afde0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 >> 10.20.193.112:0/2500092541 conn(0x7fe55c06f640 msgr2=0x7fe55c114410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 shutdown_connections 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 -- 10.20.193.112:0/2500092541 wait complete. 2026-02-24T18:33:48.248 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.245+0000 7fe561fe4640 1 librados: shutdown 2026-02-24T18:33:48.254 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 1c4e2c48-941e-4185-9331-6f37534247c8 6 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: get_monmap_and_config 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: monmap: 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:48.316289+0000 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:48.316289+0000 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.318 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 Processor -- start 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 -- start start 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: init 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient: _add_conns ranks=[2,3,1,5,0,4] 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff280073930 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff280117280 0x7ff280117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): picked mon.noname-a con 0x7ff280117280 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff280117c50 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff280128400 con 0x7ff280117c50 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2801295e0 con 0x7ff280073930 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff28012a7c0 con 0x7ff280117280 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.319 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff285885640 10 monclient(hunting): authenticate will time out at 1369.961614s 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff27e7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff280117280 0x7ff280117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.315+0000 7ff27f7fe640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7ff280117c50 0x7ff280118100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52156/0 (socket says 10.20.193.112:52156) 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27f7fe640 1 -- 10.20.193.112:0/3676198744 learned_addr learned my addr 10.20.193.112:0/3676198744 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): get_auth_request con 0x7ff280117280 auth_method 0 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.316+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.320 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 1 -- 10.20.193.112:0/3676198744 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537826205 0 0) 0x7ff2801295e0 con 0x7ff280073930 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 10 monclient(hunting): _negotiate 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 10 monclient(hunting): my global_id is 4157 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27dffb640 1 -- 10.20.193.112:0/3676198744 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff26c003620 con 0x7ff280073930 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_done global_id 4155 payload 274 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 1 -- 10.20.193.112:0/3676198744 >> v1:10.20.193.110:6789/0 conn(0x7ff280073930 legacy=0x7ff280073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 1 -- 10.20.193.112:0/3676198744 >> v1:10.20.193.32:6789/0 conn(0x7ff280117c50 legacy=0x7ff280118100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 1 monclient: found mon.noname-a 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.317+0000 7ff27e7fc640 1 -- 10.20.193.112:0/3676198744 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28012a940 con 0x7ff280117280 2026-02-24T18:33:48.321 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.318+0000 7ff27e7fc640 1 --2- 10.20.193.112:0/3676198744 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117280 0x7ff280117710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098a0 tx=0x7ff26802edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fb38a5de9d1239 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 1 -- 10.20.193.112:0/3676198744 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26803c070 con 0x7ff280117280 2026-02-24T18:33:48.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: dump: 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 1 -- 10.20.193.112:0/3676198744 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26802fae0 con 0x7ff280117280 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 1 -- 10.20.193.112:0/3676198744 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268037440 con 0x7ff280117280 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 5 monclient: authenticate success, global_id 4155 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff27dffb640 10 monclient: dump: 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.329 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 4 set_mon_vals no callback set 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 10 monclient: shutdown 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 -- 10.20.193.112:0/3676198744 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117280 msgr2=0x7ff280117710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 --2- 10.20.193.112:0/3676198744 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117280 0x7ff280117710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2680098a0 tx=0x7ff26802edd0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 -- 10.20.193.112:0/3676198744 shutdown_connections 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 --2- 10.20.193.112:0/3676198744 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117280 0x7ff280117710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 -- 10.20.193.112:0/3676198744 >> 10.20.193.112:0/3676198744 conn(0x7ff28006f640 msgr2=0x7ff280071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 -- 10.20.193.112:0/3676198744 shutdown_connections 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.320+0000 7ff285885640 1 -- 10.20.193.112:0/3676198744 wait complete. 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient: monmap: 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:48.331 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 librados: starting msgr at 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 librados: starting objecter 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 Processor -- start 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 -- start start 2026-02-24T18:33:48.332 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 librados: setting wanted keys 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 librados: calling monclient init 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient: init 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff280073930 0x7ff2801268d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): picked mon.noname-c con 0x7ff280073930 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:48.333 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff280117280 0x7ff280126f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): picked mon.noname-b con 0x7ff280117280 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): picked mon.noname-a con 0x7ff280117c50 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff280128a50 con 0x7ff280117c50 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801288d0 con 0x7ff280117280 2026-02-24T18:33:48.334 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff280128bd0 con 0x7ff280073930 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff285885640 10 monclient(hunting): authenticate will time out at 1369.967521s 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27e7fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff280117280 0x7ff280126f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27effd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff280073930 0x7ff2801268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59874/0 (socket says 10.20.193.112:59874) 2026-02-24T18:33:48.335 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 1 -- 10.20.193.112:0/694520523 learned_addr learned my addr 10.20.193.112:0/694520523 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 10 monclient(hunting): get_auth_request con 0x7ff280117c50 auth_method 0 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.321+0000 7ff27f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): get_auth_request con 0x7ff280117280 auth_method 0 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): get_auth_request con 0x7ff280073930 auth_method 0 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.336 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.322+0000 7ff27effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 10 monclient(hunting): handle_auth_done global_id 4158 payload 824 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 -- 10.20.193.112:0/694520523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff280073930 msgr2=0x7ff2801268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff280073930 0x7ff2801268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 -- 10.20.193.112:0/694520523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff280117280 msgr2=0x7ff280126f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.337 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff280117280 0x7ff280126f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 monclient: found mon.noname-a 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.323+0000 7ff27f7fe640 1 -- 10.20.193.112:0/694520523 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801b6060 con 0x7ff280117c50 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.324+0000 7ff27f7fe640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff27000cb50 tx=0x7ff27000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.324+0000 7ff257fff640 10 client.?.objecter ms_handle_connect 0x7ff280117c50 2026-02-24T18:33:48.338 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.324+0000 7ff257fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:48.341 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270017070 con 0x7ff280117c50 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: dump: 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.342 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27000ad50 con 0x7ff280117c50 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 5 monclient: authenticate success, global_id 4158 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 10 monclient: _renew_subs 2026-02-24T18:33:48.343 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801b62f0 con 0x7ff280117c50 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270012600 con 0x7ff280117c50 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff27d7fa640 4 set_mon_vals no callback set 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff257fff640 10 monclient: dump: 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.344 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 10 client.4158.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 10 monclient: _renew_subs 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801bc9e0 con 0x7ff280117c50 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.325+0000 7ff285885640 1 librados: init done 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.326+0000 7ff285885640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.326+0000 7ff285885640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.326+0000 7ff285885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.345 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.326+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff280117790 con 0x7ff280117c50 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.332+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7ff27001b3b0 con 0x7ff280117c50 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.333+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7ff27001ba60 con 0x7ff280117c50 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.333+0000 7ff257fff640 10 client.4158.objecter ms_dispatch2 0x7ff280000b90 osd_map(8..8 src has 1..8) 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.333+0000 7ff257fff640 3 client.4158.objecter handle_osd_map got epochs [8,8] > 0 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.333+0000 7ff257fff640 3 client.4158.objecter handle_osd_map decoding full epoch 8 2026-02-24T18:33:48.346 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.333+0000 7ff257fff640 20 client.4158.objecter dump_active .. 0 homeless 2026-02-24T18:33:48.374 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.371+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7ff270022080 con 0x7ff280117c50 2026-02-24T18:33:48.374 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.371+0000 7ff257fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.374 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.371+0000 7ff257fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:48.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.407+0000 7ff285885640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "1c4e2c48-941e-4185-9331-6f37534247c8", "id": 6}] 2026-02-24T18:33:48.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.407+0000 7ff285885640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "1c4e2c48-941e-4185-9331-6f37534247c8", "id": 6}] 2026-02-24T18:33:48.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.407+0000 7ff285885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.407+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1c4e2c48-941e-4185-9331-6f37534247c8", "id": 6} v 0) -- 0x7ff2801bcbb0 con 0x7ff280117c50 2026-02-24T18:33:48.482 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff257fff640 1 -- 10.20.193.112:0/694520523 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1c4e2c48-941e-4185-9331-6f37534247c8", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7ff27001b6d0 con 0x7ff280117c50 2026-02-24T18:33:48.482 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff257fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "1c4e2c48-941e-4185-9331-6f37534247c8", "id": 6}] 2026-02-24T18:33:48.482 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff257fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:48.483 INFO:teuthology.orchestra.run.trial112.stdout:6 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 10 librados: watch_flush enter 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 10 librados: watch_flush exit 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 20 client.4158.objecter shutdown clearing up homeless session... 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 10 client.4158.objecter successfully canceled tick 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 10 monclient: shutdown 2026-02-24T18:33:48.484 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 msgr2=0x7ff2801b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff27000cb50 tx=0x7ff27000a430 comp rx=0 tx=0).stop 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 shutdown_connections 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff280117280 0x7ff280126f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff280117c50 0x7ff2801b57a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 --2- 10.20.193.112:0/694520523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff280073930 0x7ff2801268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 >> 10.20.193.112:0/694520523 conn(0x7ff28006f640 msgr2=0x7ff2800705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 shutdown_connections 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 -- 10.20.193.112:0/694520523 wait complete. 2026-02-24T18:33:48.485 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.479+0000 7ff285885640 1 librados: shutdown 2026-02-24T18:33:48.491 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new ec86bc7e-386d-4547-83df-9db486b50617 7 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 10 monclient: get_monmap_and_config 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 10 monclient: monmap: 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:48.552667+0000 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:48.552667+0000 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:48.556 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 1 Processor -- start 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 1 -- start start 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.551+0000 7f4593c85640 10 monclient: init 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient: _add_conns ranks=[5,4,0,3,2,1] 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f458c11a950 0x7f458c11ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-c con 0x7f458c11a950 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f458c1151a0 0x7f458c115630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-b con 0x7f458c1151a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f458c115b70 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c122d50 con 0x7f458c115b70 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f458c123f30 con 0x7f458c1151a0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f458c1240b0 con 0x7f458c11a950 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f4593c85640 10 monclient(hunting): authenticate will time out at 1370.198238s 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45911f9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f458c1151a0 0x7f458c115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45919fa640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f458c11a950 0x7f458c11ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.557 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45921fb640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f458c115b70 0x7f458c121630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.112:52882/0 (socket says 10.20.193.112:52882) 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45921fb640 1 -- 10.20.193.112:0/2268484399 learned_addr learned my addr 10.20.193.112:0/2268484399 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45911f9640 10 monclient(hunting): get_auth_request con 0x7f458c1151a0 auth_method 0 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45911f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45911f9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45911f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45919fa640 10 monclient(hunting): get_auth_request con 0x7f458c11a950 auth_method 0 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45919fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45919fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.552+0000 7f45919fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.558 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient(hunting): handle_auth_done global_id 4163 payload 274 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 1 -- 10.20.193.112:0/2268484399 >> v2:10.20.193.32:3300/0 conn(0x7f458c1151a0 msgr2=0x7f458c115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 1 --2- 10.20.193.112:0/2268484399 >> v2:10.20.193.32:3300/0 conn(0x7f458c1151a0 0x7f458c115630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 1 -- 10.20.193.112:0/2268484399 >> v1:10.20.193.18:6789/0 conn(0x7f458c115b70 legacy=0x7f458c121630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 1 monclient: found mon.noname-c 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.553+0000 7f45919fa640 1 -- 10.20.193.112:0/2268484399 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c124230 con 0x7f458c11a950 2026-02-24T18:33:48.559 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.554+0000 7f45919fa640 1 --2- 10.20.193.112:0/2268484399 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c11a950 0x7f458c11ad30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f458c06afd0 tx=0x7f457c02ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86baaac277909d9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.563 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 1 -- 10.20.193.112:0/2268484399 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c03c070 con 0x7f458c11a950 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: dump: 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.564 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 5 monclient: authenticate success, global_id 4163 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 1 -- 10.20.193.112:0/2268484399 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c02fa60 con 0x7f458c11a950 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 1 -- 10.20.193.112:0/2268484399 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c02fe30 con 0x7f458c11a950 2026-02-24T18:33:48.565 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 4 set_mon_vals no callback set 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f45909f8640 10 monclient: dump: 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.566 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 10 monclient: shutdown 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 1 -- 10.20.193.112:0/2268484399 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c11a950 msgr2=0x7f458c11ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.556+0000 7f4593c85640 1 --2- 10.20.193.112:0/2268484399 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c11a950 0x7f458c11ad30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f458c06afd0 tx=0x7f457c02ed50 comp rx=0 tx=0).stop 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- 10.20.193.112:0/2268484399 shutdown_connections 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 --2- 10.20.193.112:0/2268484399 >> v2:10.20.193.32:3300/0 conn(0x7f458c1151a0 0x7f458c115630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 --2- 10.20.193.112:0/2268484399 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c11a950 0x7f458c11ad30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- 10.20.193.112:0/2268484399 >> 10.20.193.112:0/2268484399 conn(0x7f458c06dfb0 msgr2=0x7f458c06e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- 10.20.193.112:0/2268484399 shutdown_connections 2026-02-24T18:33:48.567 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- 10.20.193.112:0/2268484399 wait complete. 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient: monmap: 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.568 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 librados: starting msgr at 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 librados: starting objecter 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 Processor -- start 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- start start 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 librados: setting wanted keys 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 librados: calling monclient init 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient: init 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c1151a0 0x7f458c1ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-c con 0x7f458c1151a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f458c115b70 0x7f458c1abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-b con 0x7f458c115b70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:48.569 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): picked mon.noname-a con 0x7f458c11a950 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1233b0 con 0x7f458c11a950 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f458c123230 con 0x7f458c115b70 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f458c123530 con 0x7f458c1151a0 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.557+0000 7f4593c85640 10 monclient(hunting): authenticate will time out at 1370.203768s 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c1151a0 0x7f458c1ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.570 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f458c115b70 0x7f458c1abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59888/0 (socket says 10.20.193.112:59888) 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 1 -- 10.20.193.112:0/4056903206 learned_addr learned my addr 10.20.193.112:0/4056903206 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): get_auth_request con 0x7f458c11a950 auth_method 0 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): get_auth_request con 0x7f458c1151a0 auth_method 0 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): get_auth_request con 0x7f458c115b70 auth_method 0 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.571 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45919fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45921fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.558+0000 7f45911f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 10 monclient(hunting): handle_auth_done global_id 4164 payload 824 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 -- 10.20.193.112:0/4056903206 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c1151a0 msgr2=0x7f458c1ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.572 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c1151a0 0x7f458c1ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 -- 10.20.193.112:0/4056903206 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f458c115b70 msgr2=0x7f458c1abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f458c115b70 0x7f458c1abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 monclient: found mon.noname-a 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.559+0000 7f45921fb640 1 -- 10.20.193.112:0/4056903206 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c1b0940 con 0x7f458c11a950 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.560+0000 7f45921fb640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f458800cb50 tx=0x7f458800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.560+0000 7f457a7fc640 10 client.?.objecter ms_handle_connect 0x7f458c11a950 2026-02-24T18:33:48.573 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.560+0000 7f457a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588017070 con 0x7f458c11a950 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: dump: 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.576 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458800ad50 con 0x7f458c11a950 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588012600 con 0x7f458c11a950 2026-02-24T18:33:48.577 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457a7fc640 10 monclient: dump: 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.578 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 5 monclient: authenticate success, global_id 4164 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f457bfff640 4 set_mon_vals no callback set 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _renew_subs 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c1b0c30 con 0x7f458c11a950 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 client.4164.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _renew_subs 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1b84c0 con 0x7f458c11a950 2026-02-24T18:33:48.579 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 1 librados: init done 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.562+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c115630 con 0x7f458c11a950 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.568+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f45880127a0 con 0x7f458c11a950 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.569+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f458801add0 con 0x7f458c11a950 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.569+0000 7f457a7fc640 10 client.4164.objecter ms_dispatch2 0x7f458c000b90 osd_map(9..9 src has 1..9) 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.569+0000 7f457a7fc640 3 client.4164.objecter handle_osd_map got epochs [9,9] > 0 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.569+0000 7f457a7fc640 3 client.4164.objecter handle_osd_map decoding full epoch 9 2026-02-24T18:33:48.580 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.569+0000 7f457a7fc640 20 client.4164.objecter dump_active .. 0 homeless 2026-02-24T18:33:48.611 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.608+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f4588022020 con 0x7f458c11a950 2026-02-24T18:33:48.612 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.608+0000 7f457a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.612 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.608+0000 7f457a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:48.643 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.640+0000 7f4593c85640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ec86bc7e-386d-4547-83df-9db486b50617", "id": 7}] 2026-02-24T18:33:48.643 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.640+0000 7f4593c85640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ec86bc7e-386d-4547-83df-9db486b50617", "id": 7}] 2026-02-24T18:33:48.643 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.640+0000 7f4593c85640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:48.643 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.640+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec86bc7e-386d-4547-83df-9db486b50617", "id": 7} v 0) -- 0x7f458c1b8690 con 0x7f458c11a950 2026-02-24T18:33:48.673 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f457a7fc640 1 -- 10.20.193.112:0/4056903206 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec86bc7e-386d-4547-83df-9db486b50617", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f45880113a0 con 0x7f458c11a950 2026-02-24T18:33:48.673 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f457a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ec86bc7e-386d-4547-83df-9db486b50617", "id": 7}] 2026-02-24T18:33:48.673 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f457a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:48.673 INFO:teuthology.orchestra.run.trial112.stdout:7 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 10 librados: watch_flush enter 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 10 librados: watch_flush exit 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 20 client.4164.objecter shutdown clearing up homeless session... 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 10 client.4164.objecter successfully canceled tick 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 10 monclient: shutdown 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 msgr2=0x7f458c1b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f458800cb50 tx=0x7f458800a430 comp rx=0 tx=0).stop 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 shutdown_connections 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f458c115b70 0x7f458c1abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f458c11a950 0x7f458c1b0080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 --2- 10.20.193.112:0/4056903206 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f458c1151a0 0x7f458c1ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 >> 10.20.193.112:0/4056903206 conn(0x7f458c06dfb0 msgr2=0x7f458c118840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 shutdown_connections 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 -- 10.20.193.112:0/4056903206 wait complete. 2026-02-24T18:33:48.674 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.671+0000 7f4593c85640 1 librados: shutdown 2026-02-24T18:33:48.682 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 2986fffd-4bee-4a9b-847e-b217d2ac3cda 8 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: get_monmap_and_config 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: monmap: 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:48.743652+0000 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:48.743652+0000 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:48.745 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 Processor -- start 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 -- start start 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: init 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient: _add_conns ranks=[5,2,1,3,0,4] 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5148073b00 0x7f5148073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-c con 0x7f5148073b00 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5148118450 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5148118fb0 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5148121c30 con 0x7f5148118fb0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5148122e10 con 0x7f5148118450 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5148123ff0 con 0x7f5148073b00 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.742+0000 7f51505d3640 10 monclient(hunting): authenticate will time out at 1370.388959s 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514eb49640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f5148118fb0 0x7f5148120510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52166/0 (socket says 10.20.193.112:52166) 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514eb49640 1 -- 10.20.193.112:0/2276225648 learned_addr learned my addr 10.20.193.112:0/2276225648 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 1 --2- 10.20.193.112:0/2276225648 >> v2:10.20.193.110:3300/0 conn(0x7f5148073b00 0x7f5148073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): get_auth_request con 0x7f5148073b00 auth_method 0 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.743+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 1 -- 10.20.193.112:0/2276225648 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674496786 0 0) 0x7f5148121c30 con 0x7f5148118fb0 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 10 monclient(hunting): _negotiate 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 10 monclient(hunting): my global_id is 4147 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514d346640 1 -- 10.20.193.112:0/2276225648 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5130003620 con 0x7f5148118fb0 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 10 monclient(hunting): handle_auth_done global_id 4169 payload 274 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 1 -- 10.20.193.112:0/2276225648 >> v1:10.20.193.110:6789/0 conn(0x7f5148118450 legacy=0x7f51481188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 1 -- 10.20.193.112:0/2276225648 >> v1:10.20.193.32:6789/0 conn(0x7f5148118fb0 legacy=0x7f5148120510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 1 monclient: found mon.noname-c 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:48.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.744+0000 7f514e348640 1 -- 10.20.193.112:0/2276225648 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5148124170 con 0x7f5148073b00 2026-02-24T18:33:48.749 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.746+0000 7f514e348640 1 --2- 10.20.193.112:0/2276225648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148073b00 0x7f5148073ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51380098a0 tx=0x7f513802ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65dc654139873884 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.747+0000 7f514d346640 1 -- 10.20.193.112:0/2276225648 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803c070 con 0x7f5148073b00 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.747+0000 7f514d346640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: dump: 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 5 monclient: authenticate success, global_id 4169 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 1 -- 10.20.193.112:0/2276225648 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513802fa40 con 0x7f5148073b00 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 1 -- 10.20.193.112:0/2276225648 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513802fe10 con 0x7f5148073b00 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f514d346640 10 monclient: dump: 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 4 set_mon_vals no callback set 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 10 monclient: shutdown 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- 10.20.193.112:0/2276225648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148073b00 msgr2=0x7f5148073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 --2- 10.20.193.112:0/2276225648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148073b00 0x7f5148073ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51380098a0 tx=0x7f513802ed80 comp rx=0 tx=0).stop 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- 10.20.193.112:0/2276225648 shutdown_connections 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 --2- 10.20.193.112:0/2276225648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148073b00 0x7f5148073ee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- 10.20.193.112:0/2276225648 >> 10.20.193.112:0/2276225648 conn(0x7f514806f640 msgr2=0x7f5148071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:48.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- 10.20.193.112:0/2276225648 shutdown_connections 2026-02-24T18:33:48.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- 10.20.193.112:0/2276225648 wait complete. 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 10 monclient: build_initial_monmap 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 10 monclient: monmap: 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 librados: starting msgr at 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 librados: starting objecter 2026-02-24T18:33:48.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 Processor -- start 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 -- start start 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 librados: setting wanted keys 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 1 librados: calling monclient init 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.748+0000 7f51505d3640 10 monclient: init 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-b con 0x7f5148073b00 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5148118450 0x7f514811f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-a con 0x7f5148118450 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148118fb0 0x7f51481b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): picked mon.noname-c con 0x7f5148118fb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f51481234a0 con 0x7f5148118450 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5148123320 con 0x7f5148073b00 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5148123620 con 0x7f5148118fb0 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f51505d3640 10 monclient(hunting): authenticate will time out at 1370.395108s 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514eb49640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148118fb0 0x7f51481b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38756/0 (socket says 10.20.193.112:38756) 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 1 -- 10.20.193.112:0/1314273645 learned_addr learned my addr 10.20.193.112:0/1314273645 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): get_auth_request con 0x7f5148073b00 auth_method 0 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514eb49640 10 monclient(hunting): get_auth_request con 0x7f5148118fb0 auth_method 0 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514eb49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514eb49640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514eb49640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.749+0000 7f514e348640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.750+0000 7f514eb49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.750+0000 7f514eb49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.750+0000 7f514eb49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 10 monclient(hunting): handle_auth_done global_id 4150 payload 824 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 -- 10.20.193.112:0/1314273645 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148118fb0 msgr2=0x7f51481b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148118fb0 0x7f51481b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 -- 10.20.193.112:0/1314273645 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5148118450 msgr2=0x7f514811f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:48.755 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5148118450 0x7f514811f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 10 monclient: _finish_hunting 0 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 monclient: found mon.noname-b 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.751+0000 7f514e348640 1 -- 10.20.193.112:0/1314273645 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51481b35e0 con 0x7f5148073b00 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.752+0000 7f514e348640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51380099d0 tx=0x7f51380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.752+0000 7f512effd640 10 client.?.objecter ms_handle_connect 0x7f5148073b00 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.752+0000 7f512effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138046020 con 0x7f5148073b00 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:48.756 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: dump: 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: _finish_auth 0 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 5 monclient: authenticate success, global_id 4150 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _renew_subs 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51481b3810 con 0x7f5148073b00 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 client.4150.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _renew_subs 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481b3d50 con 0x7f5148073b00 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 1 librados: init done 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513802f9a0 con 0x7f5148073b00 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803c040 con 0x7f5148073b00 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f512effd640 10 monclient: dump: 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f514cb45640 4 set_mon_vals no callback set 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:48.757 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.753+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5148073ee0 con 0x7f5148073b00 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.759+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f513803f030 con 0x7f5148073b00 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.760+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5138040d10 con 0x7f5148073b00 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.760+0000 7f512effd640 10 client.4150.objecter ms_dispatch2 0x7f5148000b90 osd_map(10..10 src has 1..10) 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.760+0000 7f512effd640 3 client.4150.objecter handle_osd_map got epochs [10,10] > 0 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.760+0000 7f512effd640 3 client.4150.objecter handle_osd_map decoding full epoch 10 2026-02-24T18:33:48.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.760+0000 7f512effd640 20 client.4150.objecter dump_active .. 0 homeless 2026-02-24T18:33:48.801 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.799+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f5138050030 con 0x7f5148073b00 2026-02-24T18:33:48.801 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.799+0000 7f512effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:48.801 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.799+0000 7f512effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:48.833 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.830+0000 7f51505d3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "2986fffd-4bee-4a9b-847e-b217d2ac3cda", "id": 8}] 2026-02-24T18:33:48.833 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.830+0000 7f51505d3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "2986fffd-4bee-4a9b-847e-b217d2ac3cda", "id": 8}] 2026-02-24T18:33:48.833 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.830+0000 7f51505d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:48.833 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.830+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2986fffd-4bee-4a9b-847e-b217d2ac3cda", "id": 8} v 0) -- 0x7f5148116710 con 0x7f5148073b00 2026-02-24T18:33:48.939 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f513804a3b0 con 0x7f5148073b00 2026-02-24T18:33:48.940 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f512effd640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 0x7f511807b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:48.940 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f514db47640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 0x7f511807b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:48.940 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f514db47640 10 monclient: get_auth_request con 0x7f51180791e0 auth_method 0 2026-02-24T18:33:48.940 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f514db47640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 0x7f511807b5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f513c006fd0 tx=0x7f513c006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:48.940 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:48.936+0000 7f512effd640 10 client.4150.objecter ms_handle_connect 0x7f51180791e0 2026-02-24T18:33:48.976 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:48.974+0000 7f81bf7e5640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-24T18:33:48.977 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:33:48.974+0000 7f81bf7e5640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-24T18:33:49.028 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f512effd640 1 -- 10.20.193.112:0/1314273645 <== mon.1 v2:10.20.193.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2986fffd-4bee-4a9b-847e-b217d2ac3cda", "id": 8}]=0 v11) ==== 112+0+2 (secure 0 0 0) 0x7f5148116710 con 0x7f5148073b00 2026-02-24T18:33:49.028 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f512effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "2986fffd-4bee-4a9b-847e-b217d2ac3cda", "id": 8}] 2026-02-24T18:33:49.028 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f512effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:49.028 INFO:teuthology.orchestra.run.trial112.stdout:8 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 10 librados: watch_flush enter 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 10 librados: watch_flush exit 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 20 client.4150.objecter shutdown clearing up homeless session... 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 10 client.4150.objecter successfully canceled tick 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 msgr2=0x7f511807b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 0x7f511807b5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f513c006fd0 tx=0x7f513c006d40 comp rx=0 tx=0).stop 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 10 monclient: shutdown 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 msgr2=0x7f514811ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.029 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51380099d0 tx=0x7f51380047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 shutdown_connections 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f51180791e0 0x7f511807b5d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5148118fb0 0x7f51481b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5148118450 0x7f514811f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 --2- 10.20.193.112:0/1314273645 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5148073b00 0x7f514811ed90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 >> 10.20.193.112:0/1314273645 conn(0x7f514806f640 msgr2=0x7f5148072470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.025+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 shutdown_connections 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.026+0000 7f51505d3640 1 -- 10.20.193.112:0/1314273645 wait complete. 2026-02-24T18:33:49.030 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.026+0000 7f51505d3640 1 librados: shutdown 2026-02-24T18:33:49.035 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 16e6895e-d37b-4a74-be40-45204e7d1077 9 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: get_monmap_and_config 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: monmap: 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:49.096453+0000 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:49.096453+0000 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:49.098 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 Processor -- start 2026-02-24T18:33:49.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 -- start start 2026-02-24T18:33:49.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: init 2026-02-24T18:33:49.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient: _add_conns ranks=[5,3,2,1,4,0] 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 0x7faf48073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-c con 0x7faf48073b00 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7faf48118450 0x7faf481188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-a con 0x7faf48118450 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faf48118de0 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf48074600 con 0x7faf48118de0 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7faf48121c30 con 0x7faf48118450 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7faf48121db0 con 0x7faf48073b00 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.095+0000 7faf4e1fe640 10 monclient(hunting): authenticate will time out at 1370.741776s 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 0x7faf48073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7faf48118450 0x7faf481188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 0x7faf48073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:52178/0 (socket says 10.20.193.112:52178) 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 1 -- 10.20.193.112:0/1021020225 learned_addr learned my addr 10.20.193.112:0/1021020225 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): get_auth_request con 0x7faf48073b00 auth_method 0 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): get_auth_request con 0x7faf48118450 auth_method 0 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.103 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.096+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 10 monclient(hunting): handle_auth_done global_id 4167 payload 274 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 1 -- 10.20.193.112:0/1021020225 >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 msgr2=0x7faf48073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 1 --2- 10.20.193.112:0/1021020225 >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 0x7faf48073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 1 -- 10.20.193.112:0/1021020225 >> v1:10.20.193.110:6789/0 conn(0x7faf48118de0 legacy=0x7faf4811f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.104 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 1 monclient: found mon.noname-a 2026-02-24T18:33:49.105 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.105 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.105 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.097+0000 7faf46ffd640 1 -- 10.20.193.112:0/1021020225 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf48121f30 con 0x7faf48118450 2026-02-24T18:33:49.105 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.098+0000 7faf46ffd640 1 --2- 10.20.193.112:0/1021020225 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481188a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf4806bd50 tx=0x7faf3802ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ec72d88320ae57 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.111 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 1 -- 10.20.193.112:0/1021020225 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3803c070 con 0x7faf48118450 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: dump: 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.112 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 1 -- 10.20.193.112:0/1021020225 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3802faf0 con 0x7faf48118450 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 5 monclient: authenticate success, global_id 4167 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 1 -- 10.20.193.112:0/1021020225 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38037440 con 0x7faf48118450 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:49.113 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 4 set_mon_vals no callback set 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf467fc640 10 monclient: dump: 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.114 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 10 monclient: shutdown 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 -- 10.20.193.112:0/1021020225 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 msgr2=0x7faf481188a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/1021020225 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481188a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf4806bd50 tx=0x7faf3802ede0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 -- 10.20.193.112:0/1021020225 shutdown_connections 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/1021020225 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481188a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/1021020225 >> v2:10.20.193.110:3300/0 conn(0x7faf48073b00 0x7faf48073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 -- 10.20.193.112:0/1021020225 >> 10.20.193.112:0/1021020225 conn(0x7faf4806f640 msgr2=0x7faf48071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 -- 10.20.193.112:0/1021020225 shutdown_connections 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 1 -- 10.20.193.112:0/1021020225 wait complete. 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.115 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.100+0000 7faf4e1fe640 10 monclient: monmap: 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 librados: starting msgr at 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 librados: starting objecter 2026-02-24T18:33:49.116 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 Processor -- start 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 -- start start 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 librados: setting wanted keys 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 librados: calling monclient init 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient: init 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faf48073b00 0x7faf481b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-b con 0x7faf48073b00 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-a con 0x7faf48118450 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faf48118de0 0x7faf481ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): picked mon.noname-c con 0x7faf48118de0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:49.117 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7faf481210e0 con 0x7faf48118450 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf48120f60 con 0x7faf48073b00 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf48121260 con 0x7faf48118de0 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf4e1fe640 10 monclient(hunting): authenticate will time out at 1370.747393s 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf47fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faf48118de0 0x7faf481ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf477fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faf48073b00 0x7faf481b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59908/0 (socket says 10.20.193.112:59908) 2026-02-24T18:33:49.118 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 1 -- 10.20.193.112:0/597692859 learned_addr learned my addr 10.20.193.112:0/597692859 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 10 monclient(hunting): get_auth_request con 0x7faf48118450 auth_method 0 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf46ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf477fe640 10 monclient(hunting): get_auth_request con 0x7faf48073b00 auth_method 0 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf477fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf47fff640 10 monclient(hunting): get_auth_request con 0x7faf48118de0 auth_method 0 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf47fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.119 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.101+0000 7faf47fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf46ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.102+0000 7faf47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 10 monclient(hunting): handle_auth_done global_id 4170 payload 824 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 -- 10.20.193.112:0/597692859 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faf48118de0 msgr2=0x7faf481ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faf48118de0 0x7faf481ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.120 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 -- 10.20.193.112:0/597692859 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faf48073b00 msgr2=0x7faf481b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faf48073b00 0x7faf481b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 monclient: found mon.noname-a 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.103+0000 7faf46ffd640 1 -- 10.20.193.112:0/597692859 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf481abe60 con 0x7faf48118450 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.104+0000 7faf46ffd640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf38009130 tx=0x7faf380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.104+0000 7faf23fff640 10 client.?.objecter ms_handle_connect 0x7faf48118450 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.104+0000 7faf23fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3803c040 con 0x7faf48118450 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.121 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: dump: 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.122 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 5 monclient: authenticate success, global_id 4170 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _renew_subs 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf481ac0f0 con 0x7faf48118450 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 client.4170.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _renew_subs 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf481b82c0 con 0x7faf48118450 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 1 librados: init done 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.123 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf48073ee0 con 0x7faf48118450 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3802fa70 con 0x7faf48118450 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38042810 con 0x7faf48118450 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf23fff640 10 monclient: dump: 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.124 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.125 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.125 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.125 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.125 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.125 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.105+0000 7faf45ffb640 4 set_mon_vals no callback set 2026-02-24T18:33:49.214 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.211+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7faf3804b450 con 0x7faf48118450 2026-02-24T18:33:49.214 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.211+0000 7faf23fff640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 0x7faf1807b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.214 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.212+0000 7faf477fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 0x7faf1807b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.214 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.212+0000 7faf477fe640 10 monclient: get_auth_request con 0x7faf18078c50 auth_method 0 2026-02-24T18:33:49.214 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.212+0000 7faf477fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 0x7faf1807b110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf34006dc0 tx=0x7faf34009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.215 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.212+0000 7faf23fff640 10 client.4170.objecter ms_handle_connect 0x7faf18078c50 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.218+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1760+0+0 (secure 0 0 0) 0x7faf380827e0 con 0x7faf48118450 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.218+0000 7faf23fff640 10 client.4170.objecter ms_dispatch2 0x7faf48000b90 osd_map(11..11 src has 1..11) 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.218+0000 7faf23fff640 3 client.4170.objecter handle_osd_map got epochs [11,11] > 0 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.218+0000 7faf23fff640 3 client.4170.objecter handle_osd_map decoding full epoch 11 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.218+0000 7faf23fff640 20 client.4170.objecter dump_active .. 0 homeless 2026-02-24T18:33:49.221 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.219+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faf380867a0 con 0x7faf48118450 2026-02-24T18:33:49.222 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.219+0000 7faf23fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.222 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.219+0000 7faf23fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:49.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.306+0000 7faf4e1fe640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "16e6895e-d37b-4a74-be40-45204e7d1077", "id": 9}] 2026-02-24T18:33:49.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.306+0000 7faf4e1fe640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "16e6895e-d37b-4a74-be40-45204e7d1077", "id": 9}] 2026-02-24T18:33:49.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.306+0000 7faf4e1fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:49.309 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.306+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "16e6895e-d37b-4a74-be40-45204e7d1077", "id": 9} v 0) -- 0x7faf48116b20 con 0x7faf48118450 2026-02-24T18:33:49.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf23fff640 1 -- 10.20.193.112:0/597692859 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16e6895e-d37b-4a74-be40-45204e7d1077", "id": 9}]=0 v12) ==== 112+0+2 (secure 0 0 0) 0x7faf3808aaa0 con 0x7faf48118450 2026-02-24T18:33:49.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf23fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "16e6895e-d37b-4a74-be40-45204e7d1077", "id": 9}] 2026-02-24T18:33:49.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf23fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stdout:9 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 10 librados: watch_flush enter 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 10 librados: watch_flush exit 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 20 client.4170.objecter shutdown clearing up homeless session... 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 10 client.4170.objecter successfully canceled tick 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 msgr2=0x7faf1807b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 0x7faf1807b110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf34006dc0 tx=0x7faf34009340 comp rx=0 tx=0).stop 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 10 monclient: shutdown 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.327 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 msgr2=0x7faf481b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf38009130 tx=0x7faf380047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 shutdown_connections 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faf18078c50 0x7faf1807b110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faf48118de0 0x7faf481ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faf48118450 0x7faf481b1b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 --2- 10.20.193.112:0/597692859 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faf48073b00 0x7faf481b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 >> 10.20.193.112:0/597692859 conn(0x7faf4806f640 msgr2=0x7faf48072550 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 shutdown_connections 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.324+0000 7faf4e1fe640 1 -- 10.20.193.112:0/597692859 wait complete. 2026-02-24T18:33:49.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.325+0000 7faf4e1fe640 1 librados: shutdown 2026-02-24T18:33:49.334 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 8d89aa64-bc98-4a27-8ece-99b034a9095c 10 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: get_monmap_and_config 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: monmap: 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:49.394487+0000 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:49.394487+0000 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:49.396 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 Processor -- start 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 -- start start 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: init 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient: _add_conns ranks=[5,3,4,1,2,0] 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f7690073b00 0x7f7690073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-c con 0x7f7690073b00 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7690118450 0x7f76901188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-a con 0x7f7690118450 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 0x7f769011f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-b con 0x7f7690118de0 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7690074600 con 0x7f7690118450 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f769011fa40 con 0x7f7690118de0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f769011fbc0 con 0x7f7690073b00 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.393+0000 7f7696ebf640 10 monclient(hunting): authenticate will time out at 1371.039796s 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 0x7f769011f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f7690073b00 0x7f7690073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7690118450 0x7f76901188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 0x7f769011f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38774/0 (socket says 10.20.193.112:38774) 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 1 -- 10.20.193.112:0/1839639959 learned_addr learned my addr 10.20.193.112:0/1839639959 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:49.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): get_auth_request con 0x7f7690118de0 auth_method 0 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): get_auth_request con 0x7f7690073b00 auth_method 0 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): get_auth_request con 0x7f7690118450 auth_method 0 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.394+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 10 monclient(hunting): handle_auth_done global_id 4181 payload 274 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 -- 10.20.193.112:0/1839639959 >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 msgr2=0x7f769011f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 --2- 10.20.193.112:0/1839639959 >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 0x7f769011f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 -- 10.20.193.112:0/1839639959 >> v2:10.20.193.18:3300/0 conn(0x7f7690118450 msgr2=0x7f76901188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 --2- 10.20.193.112:0/1839639959 >> v2:10.20.193.18:3300/0 conn(0x7f7690118450 0x7f76901188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 monclient: found mon.noname-c 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.398 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.395+0000 7f7694c34640 1 -- 10.20.193.112:0/1839639959 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769011fd40 con 0x7f7690073b00 2026-02-24T18:33:49.399 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.396+0000 7f7694c34640 1 --2- 10.20.193.112:0/1839639959 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f7690073ee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76900745d0 tx=0x7f768002edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1de761e20e65a6 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 1 -- 10.20.193.112:0/1839639959 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768003c070 con 0x7f7690073b00 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: dump: 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.400 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 1 -- 10.20.193.112:0/1839639959 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768002fa80 con 0x7f7690073b00 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 1 -- 10.20.193.112:0/1839639959 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768002fe50 con 0x7f7690073b00 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f76877fe640 10 monclient: dump: 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f7696ebf640 5 monclient: authenticate success, global_id 4181 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f7696ebf640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:49.401 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.397+0000 7f7696ebf640 4 set_mon_vals no callback set 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 10 monclient: shutdown 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- 10.20.193.112:0/1839639959 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 msgr2=0x7f7690073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1839639959 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f7690073ee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76900745d0 tx=0x7f768002edc0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- 10.20.193.112:0/1839639959 shutdown_connections 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1839639959 >> v2:10.20.193.32:3300/0 conn(0x7f7690118de0 0x7f769011f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1839639959 >> v2:10.20.193.18:3300/0 conn(0x7f7690118450 0x7f76901188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1839639959 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f7690073ee0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- 10.20.193.112:0/1839639959 >> 10.20.193.112:0/1839639959 conn(0x7f769006f640 msgr2=0x7f7690071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- 10.20.193.112:0/1839639959 shutdown_connections 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- 10.20.193.112:0/1839639959 wait complete. 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 10 monclient: monmap: 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:49.402 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 librados: starting msgr at 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 librados: starting objecter 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 Processor -- start 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 -- start start 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 librados: setting wanted keys 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 1 librados: calling monclient init 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.398+0000 7f7696ebf640 10 monclient: init 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-c con 0x7f7690073b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7690118450 0x7f76901af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-b con 0x7f7690118450 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7690118de0 0x7f76901a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): picked mon.noname-a con 0x7f7690118de0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76901203c0 con 0x7f7690118de0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7690120240 con 0x7f7690118450 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7690120540 con 0x7f7690073b00 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7696ebf640 10 monclient(hunting): authenticate will time out at 1371.045144s 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.112:52214/0 (socket says 10.20.193.112:52214) 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 1 -- 10.20.193.112:0/1995069523 learned_addr learned my addr 10.20.193.112:0/1995069523 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): get_auth_request con 0x7f7690073b00 auth_method 0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7687fff640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7690118450 0x7f76901af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7695435640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7690118de0 0x7f76901a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7687fff640 10 monclient(hunting): get_auth_request con 0x7f7690118450 auth_method 0 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7687fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7687fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7687fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7694c34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.403 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7695435640 10 monclient(hunting): get_auth_request con 0x7f7690118de0 auth_method 0 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7695435640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7695435640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.399+0000 7f7695435640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7687fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.400+0000 7f7695435640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 10 monclient(hunting): handle_auth_done global_id 4184 payload 824 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 -- 10.20.193.112:0/1995069523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7690118450 msgr2=0x7f76901af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7690118450 0x7f76901af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 -- 10.20.193.112:0/1995069523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7690118de0 msgr2=0x7f76901a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7690118de0 0x7f76901a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 monclient: found mon.noname-c 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.401+0000 7f7694c34640 1 -- 10.20.193.112:0/1995069523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76901a9b40 con 0x7f7690073b00 2026-02-24T18:33:49.404 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.402+0000 7f7694c34640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7680009130 tx=0x7f76800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.405 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.402+0000 7f76857fa640 10 client.?.objecter ms_handle_connect 0x7f7690073b00 2026-02-24T18:33:49.405 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.402+0000 7f76857fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768003c040 con 0x7f7690073b00 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: dump: 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.406 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7680042430 con 0x7f7690073b00 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 5 monclient: authenticate success, global_id 4184 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 10 monclient: _renew_subs 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76901a9d70 con 0x7f7690073b00 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680042820 con 0x7f7690073b00 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7686ffd640 4 set_mon_vals no callback set 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f76857fa640 10 monclient: dump: 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 10 client.4184.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 10 monclient: _renew_subs 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76901b5f80 con 0x7f7690073b00 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.403+0000 7f7696ebf640 1 librados: init done 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.404+0000 7f7696ebf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.404+0000 7f7696ebf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.404+0000 7f7696ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.407 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.404+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7690073ee0 con 0x7f7690073b00 2026-02-24T18:33:49.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.415+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f76800429c0 con 0x7f7690073b00 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.415+0000 7f76857fa640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 0x7f766007b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.415+0000 7f7687fff640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 0x7f766007b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.415+0000 7f7687fff640 10 monclient: get_auth_request con 0x7f7660078ca0 auth_method 0 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 10 client.4184.objecter ms_handle_connect 0x7f7660078ca0 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f7687fff640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 0x7f766007b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76901aed60 tx=0x7f7678009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 1875+0+0 (secure 0 0 0) 0x7f7680042ce0 con 0x7f7690073b00 2026-02-24T18:33:49.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 10 client.4184.objecter ms_dispatch2 0x7f7690000b90 osd_map(12..12 src has 1..12) 2026-02-24T18:33:49.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 3 client.4184.objecter handle_osd_map got epochs [12,12] > 0 2026-02-24T18:33:49.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 3 client.4184.objecter handle_osd_map decoding full epoch 12 2026-02-24T18:33:49.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.416+0000 7f76857fa640 20 client.4184.objecter dump_active .. 0 homeless 2026-02-24T18:33:49.517 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.513+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f76800876e0 con 0x7f7690073b00 2026-02-24T18:33:49.517 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.513+0000 7f76857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.517 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.513+0000 7f76857fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:49.603 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.600+0000 7f7696ebf640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8d89aa64-bc98-4a27-8ece-99b034a9095c", "id": 10}] 2026-02-24T18:33:49.603 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.600+0000 7f7696ebf640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8d89aa64-bc98-4a27-8ece-99b034a9095c", "id": 10}] 2026-02-24T18:33:49.603 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.600+0000 7f7696ebf640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:49.603 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.600+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d89aa64-bc98-4a27-8ece-99b034a9095c", "id": 10} v 0) -- 0x7f769011a5c0 con 0x7f7690073b00 2026-02-24T18:33:49.647 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f76857fa640 1 -- 10.20.193.112:0/1995069523 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d89aa64-bc98-4a27-8ece-99b034a9095c", "id": 10}]=0 v13) ==== 113+0+3 (secure 0 0 0) 0x7f768008b9e0 con 0x7f7690073b00 2026-02-24T18:33:49.647 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f76857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8d89aa64-bc98-4a27-8ece-99b034a9095c", "id": 10}] 2026-02-24T18:33:49.648 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f76857fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:49.648 INFO:teuthology.orchestra.run.trial112.stdout:10 2026-02-24T18:33:49.649 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 10 librados: watch_flush enter 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 10 librados: watch_flush exit 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 20 client.4184.objecter shutdown clearing up homeless session... 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 10 client.4184.objecter successfully canceled tick 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 msgr2=0x7f766007b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 0x7f766007b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76901aed60 tx=0x7f7678009290 comp rx=0 tx=0).stop 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 10 monclient: shutdown 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 msgr2=0x7f76901af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7680009130 tx=0x7f76800047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.650 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 shutdown_connections 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7660078ca0 0x7f766007b160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7690118450 0x7f76901af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7690118de0 0x7f76901a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 --2- 10.20.193.112:0/1995069523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7690073b00 0x7f76901af1d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 >> 10.20.193.112:0/1995069523 conn(0x7f769006f640 msgr2=0x7f7690117460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 shutdown_connections 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 -- 10.20.193.112:0/1995069523 wait complete. 2026-02-24T18:33:49.651 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.645+0000 7f7696ebf640 1 librados: shutdown 2026-02-24T18:33:49.657 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 9a649a86-160c-47fc-bcdf-ca545fc0d62a 11 2026-02-24T18:33:49.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.717+0000 7f5dec085640 10 monclient: get_monmap_and_config 2026-02-24T18:33:49.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.717+0000 7f5dec085640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.730 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.717+0000 7f5dec085640 10 monclient: monmap: 2026-02-24T18:33:49.730 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:49.718916+0000 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:49.718916+0000 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:49.731 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 Processor -- start 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 -- start start 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient: init 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient: _add_conns ranks=[4,2,3,0,1,5] 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f5de4073b00 0x7f5de4073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-b con 0x7f5de4073b00 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5de4118450 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f5de4118fb0 0x7f5de4120510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-a con 0x7f5de4118fb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:49.732 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de4120a50 con 0x7f5de4118450 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5de4121c30 con 0x7f5de4118fb0 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5de4121db0 con 0x7f5de4073b00 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dec085640 10 monclient(hunting): authenticate will time out at 1371.364298s 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de9dfa640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f5de4073b00 0x7f5de4073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dea5fb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f5de4118fb0 0x7f5de4120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de95f9640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f5de4118450 0x7f5de41188a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.112:51196/0 (socket says 10.20.193.112:51196) 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de95f9640 1 -- 10.20.193.112:0/281278889 learned_addr learned my addr 10.20.193.112:0/281278889 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de9dfa640 10 monclient(hunting): get_auth_request con 0x7f5de4073b00 auth_method 0 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de9dfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.733 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de9dfa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5de9dfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dea5fb640 10 monclient(hunting): get_auth_request con 0x7f5de4118fb0 auth_method 0 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dea5fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dea5fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.718+0000 7f5dea5fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_done global_id 4162 payload 274 2026-02-24T18:33:49.734 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 1 -- 10.20.193.112:0/281278889 >> v2:10.20.193.18:3300/0 conn(0x7f5de4118fb0 msgr2=0x7f5de4120510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 1 --2- 10.20.193.112:0/281278889 >> v2:10.20.193.18:3300/0 conn(0x7f5de4118fb0 0x7f5de4120510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 1 -- 10.20.193.112:0/281278889 >> v1:10.20.193.110:6789/0 conn(0x7f5de4118450 legacy=0x7f5de41188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 1 monclient: found mon.noname-b 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.719+0000 7f5de9dfa640 1 -- 10.20.193.112:0/281278889 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de4121f30 con 0x7f5de4073b00 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.720+0000 7f5de9dfa640 1 --2- 10.20.193.112:0/281278889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4073b00 0x7f5de4073ee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5de406bd50 tx=0x7f5dd402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19a3dcdd43638c7b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 1 -- 10.20.193.112:0/281278889 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd403c070 con 0x7f5de4073b00 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:49.735 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: dump: 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.736 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 5 monclient: authenticate success, global_id 4162 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 1 -- 10.20.193.112:0/281278889 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd402fa60 con 0x7f5de4073b00 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.737 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 1 -- 10.20.193.112:0/281278889 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd402fe30 con 0x7f5de4073b00 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 4 set_mon_vals no callback set 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5de8df8640 10 monclient: dump: 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.738 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 10 monclient: shutdown 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 1 -- 10.20.193.112:0/281278889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4073b00 msgr2=0x7f5de4073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.722+0000 7f5dec085640 1 --2- 10.20.193.112:0/281278889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4073b00 0x7f5de4073ee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5de406bd50 tx=0x7f5dd402ed50 comp rx=0 tx=0).stop 2026-02-24T18:33:49.739 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- 10.20.193.112:0/281278889 shutdown_connections 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 --2- 10.20.193.112:0/281278889 >> v2:10.20.193.18:3300/0 conn(0x7f5de4118fb0 0x7f5de4120510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 --2- 10.20.193.112:0/281278889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4073b00 0x7f5de4073ee0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- 10.20.193.112:0/281278889 >> 10.20.193.112:0/281278889 conn(0x7f5de406f640 msgr2=0x7f5de4071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- 10.20.193.112:0/281278889 shutdown_connections 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- 10.20.193.112:0/281278889 wait complete. 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient: build_initial_monmap 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient: monmap: 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:49.740 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 librados: starting msgr at 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 librados: starting objecter 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 Processor -- start 2026-02-24T18:33:49.741 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- start start 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 librados: setting wanted keys 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 librados: calling monclient init 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient: init 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:33:49.742 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5de4073b00 0x7f5de41ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-c con 0x7f5de4073b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-b con 0x7f5de4118450 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5de4118fb0 0x7f5de41b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): picked mon.noname-a con 0x7f5de4118fb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41210e0 con 0x7f5de4118fb0 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4120f60 con 0x7f5de4118450 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4121260 con 0x7f5de4073b00 2026-02-24T18:33:49.743 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.723+0000 7f5dec085640 10 monclient(hunting): authenticate will time out at 1371.369820s 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5de4118fb0 0x7f5de41b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38800/0 (socket says 10.20.193.112:38800) 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5de4073b00 0x7f5de41ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 1 -- 10.20.193.112:0/3951346426 learned_addr learned my addr 10.20.193.112:0/3951346426 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): get_auth_request con 0x7f5de4118450 auth_method 0 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): get_auth_request con 0x7f5de4118fb0 auth_method 0 2026-02-24T18:33:49.744 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de95f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): get_auth_request con 0x7f5de4073b00 auth_method 0 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.745 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5dea5fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.724+0000 7f5de9dfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 10 monclient(hunting): handle_auth_done global_id 4165 payload 824 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 -- 10.20.193.112:0/3951346426 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5de4073b00 msgr2=0x7f5de41ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5de4073b00 0x7f5de41ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 -- 10.20.193.112:0/3951346426 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5de4118fb0 msgr2=0x7f5de41b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5de4118fb0 0x7f5de41b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 10 monclient: _finish_hunting 0 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 monclient: found mon.noname-b 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.746 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.725+0000 7f5de95f9640 1 -- 10.20.193.112:0/3951346426 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41b0970 con 0x7f5de4118450 2026-02-24T18:33:49.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.726+0000 7f5de95f9640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c730 tx=0x7f5dd800cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.726+0000 7f5dd27fc640 10 client.?.objecter ms_handle_connect 0x7f5de4118450 2026-02-24T18:33:49.747 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.726+0000 7f5dd27fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8017070 con 0x7f5de4118450 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: dump: 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.750 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd8005b90 con 0x7f5de4118450 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8007070 con 0x7f5de4118450 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:49.751 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd3fff640 4 set_mon_vals no callback set 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dd27fc640 10 monclient: dump: 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:49.752 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 5 monclient: authenticate success, global_id 4165 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _renew_subs 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41b0c60 con 0x7f5de4118450 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 client.4165.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _renew_subs 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41b84f0 con 0x7f5de4118450 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 1 librados: init done 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.753 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.728+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de4073ee0 con 0x7f5de4118450 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.739+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f5dd8007210 con 0x7f5de4118450 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.739+0000 7f5dd27fc640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 0x7f5db407b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.739+0000 7f5de9dfa640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 0x7f5db407b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.739+0000 7f5de9dfa640 10 monclient: get_auth_request con 0x7f5db4078c50 auth_method 0 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 10 client.4165.objecter ms_handle_connect 0x7f5db4078c50 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5de9dfa640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 0x7f5db407b110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5de406bd50 tx=0x7f5dd4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 1990+0+0 (secure 0 0 0) 0x7f5dd805d400 con 0x7f5de4118450 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 10 client.4165.objecter ms_dispatch2 0x7f5de4000b90 osd_map(13..13 src has 1..13) 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 3 client.4165.objecter handle_osd_map got epochs [13,13] > 0 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 3 client.4165.objecter handle_osd_map decoding full epoch 13 2026-02-24T18:33:49.754 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.740+0000 7f5dd27fc640 20 client.4165.objecter dump_active .. 0 homeless 2026-02-24T18:33:49.840 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.838+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5dd8021020 con 0x7f5de4118450 2026-02-24T18:33:49.841 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.838+0000 7f5dd27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:49.841 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.838+0000 7f5dd27fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:49.927 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.924+0000 7f5dec085640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9a649a86-160c-47fc-bcdf-ca545fc0d62a", "id": 11}] 2026-02-24T18:33:49.927 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.924+0000 7f5dec085640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9a649a86-160c-47fc-bcdf-ca545fc0d62a", "id": 11}] 2026-02-24T18:33:49.927 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.924+0000 7f5dec085640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:49.927 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.924+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9a649a86-160c-47fc-bcdf-ca545fc0d62a", "id": 11} v 0) -- 0x7f5de4114310 con 0x7f5de4118450 2026-02-24T18:33:49.996 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dd27fc640 1 -- 10.20.193.112:0/3951346426 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a649a86-160c-47fc-bcdf-ca545fc0d62a", "id": 11}]=0 v14) ==== 113+0+3 (secure 0 0 0) 0x7f5dd80613c0 con 0x7f5de4118450 2026-02-24T18:33:49.996 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dd27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9a649a86-160c-47fc-bcdf-ca545fc0d62a", "id": 11}] 2026-02-24T18:33:49.996 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dd27fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:49.997 INFO:teuthology.orchestra.run.trial112.stdout:11 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 10 librados: watch_flush enter 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 10 librados: watch_flush exit 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 20 client.4165.objecter shutdown clearing up homeless session... 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 10 client.4165.objecter successfully canceled tick 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 msgr2=0x7f5db407b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.998 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 0x7f5db407b110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5de406bd50 tx=0x7f5dd4033000 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 10 monclient: shutdown 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 msgr2=0x7f5de41abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c730 tx=0x7f5dd800cc00 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 shutdown_connections 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5db4078c50 0x7f5db407b110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5de4118450 0x7f5de41abef0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5de4118fb0 0x7f5de41b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 --2- 10.20.193.112:0/3951346426 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5de4073b00 0x7f5de41ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 >> 10.20.193.112:0/3951346426 conn(0x7f5de406f640 msgr2=0x7f5de4072420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 shutdown_connections 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 -- 10.20.193.112:0/3951346426 wait complete. 2026-02-24T18:33:49.999 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:49.993+0000 7f5dec085640 1 librados: shutdown 2026-02-24T18:33:50.006 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 53546e48-a93a-4f5f-a114-9ea08210aba6 12 2026-02-24T18:33:50.069 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.066+0000 7fb688355640 10 monclient: get_monmap_and_config 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.066+0000 7fb688355640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.066+0000 7fb688355640 10 monclient: monmap: 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:50.067887+0000 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:50.067887+0000 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:50.070 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:50.071 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:50.071 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:50.071 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:50.071 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 Processor -- start 2026-02-24T18:33:50.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 -- start start 2026-02-24T18:33:50.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient: init 2026-02-24T18:33:50.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient: _add_conns ranks=[3,5,4,2,0,1] 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb680114a50 0x7fb680114e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): picked mon.noname-a con 0x7fb680114a50 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fb6801193d0 0x7fb680119860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): picked mon.noname-c con 0x7fb6801193d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 0x7fb6801216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): picked mon.noname-b con 0x7fb680119da0 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb680121c00 con 0x7fb680114a50 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb680121d80 con 0x7fb680119da0 2026-02-24T18:33:50.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb680121f00 con 0x7fb6801193d0 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb688355640 10 monclient(hunting): authenticate will time out at 1371.713323s 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 0x7fb6801216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6858c9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fb6801193d0 0x7fb680119860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6860ca640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb680114a50 0x7fb680114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 0x7fb6801216c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38816/0 (socket says 10.20.193.112:38816) 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 1 -- 10.20.193.112:0/3586178685 learned_addr learned my addr 10.20.193.112:0/3586178685 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 10 monclient(hunting): get_auth_request con 0x7fb680119da0 auth_method 0 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.067+0000 7fb6868cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): get_auth_request con 0x7fb680114a50 auth_method 0 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): get_auth_request con 0x7fb6801193d0 auth_method 0 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.068+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 10 monclient(hunting): handle_auth_done global_id 4185 payload 274 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 -- 10.20.193.112:0/3586178685 >> v2:10.20.193.110:3300/0 conn(0x7fb6801193d0 msgr2=0x7fb680119860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 --2- 10.20.193.112:0/3586178685 >> v2:10.20.193.110:3300/0 conn(0x7fb6801193d0 0x7fb680119860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 -- 10.20.193.112:0/3586178685 >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 msgr2=0x7fb6801216c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 --2- 10.20.193.112:0/3586178685 >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 0x7fb6801216c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 monclient: found mon.noname-a 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.069+0000 7fb6860ca640 1 -- 10.20.193.112:0/3586178685 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb680122080 con 0x7fb680114a50 2026-02-24T18:33:50.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.070+0000 7fb6860ca640 1 --2- 10.20.193.112:0/3586178685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb680114e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb680115490 tx=0x7fb67002ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36aa7e2d0c023c9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 1 -- 10.20.193.112:0/3586178685 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67003c070 con 0x7fb680114a50 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: dump: 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.086 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb688355640 5 monclient: authenticate success, global_id 4185 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb688355640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 1 -- 10.20.193.112:0/3586178685 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67002f9f0 con 0x7fb680114a50 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 1 -- 10.20.193.112:0/3586178685 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67002fdc0 con 0x7fb680114a50 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb6850c8640 10 monclient: dump: 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb688355640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:50.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.071+0000 7fb688355640 4 set_mon_vals no callback set 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: shutdown 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- 10.20.193.112:0/3586178685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 msgr2=0x7fb680114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- 10.20.193.112:0/3586178685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb680114e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb680115490 tx=0x7fb67002ed30 comp rx=0 tx=0).stop 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- 10.20.193.112:0/3586178685 shutdown_connections 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- 10.20.193.112:0/3586178685 >> v2:10.20.193.32:3300/0 conn(0x7fb680119da0 0x7fb6801216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- 10.20.193.112:0/3586178685 >> v2:10.20.193.110:3300/0 conn(0x7fb6801193d0 0x7fb680119860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- 10.20.193.112:0/3586178685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb680114e30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- 10.20.193.112:0/3586178685 >> 10.20.193.112:0/3586178685 conn(0x7fb68006dd80 msgr2=0x7fb68006e190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- 10.20.193.112:0/3586178685 shutdown_connections 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- 10.20.193.112:0/3586178685 wait complete. 2026-02-24T18:33:50.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: monmap: 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 librados: starting msgr at 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 librados: starting objecter 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 Processor -- start 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- start start 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 librados: setting wanted keys 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 librados: calling monclient init 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: init 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb6801b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): picked mon.noname-a con 0x7fb680114a50 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 0x7fb6801b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): picked mon.noname-b con 0x7fb6801193d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb680119da0 0x7fb6801ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): picked mon.noname-c con 0x7fb680119da0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb680122640 con 0x7fb680114a50 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801224c0 con 0x7fb6801193d0 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801227c0 con 0x7fb680119da0 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.072+0000 7fb688355640 10 monclient(hunting): authenticate will time out at 1371.718852s 2026-02-24T18:33:50.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb6801b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 0x7fb6801b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb6801b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59968/0 (socket says 10.20.193.112:59968) 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 1 -- 10.20.193.112:0/3042398139 learned_addr learned my addr 10.20.193.112:0/3042398139 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): get_auth_request con 0x7fb680114a50 auth_method 0 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb680119da0 0x7fb6801ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): get_auth_request con 0x7fb6801193d0 auth_method 0 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): get_auth_request con 0x7fb680119da0 auth_method 0 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6860ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6858c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.073+0000 7fb6868cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 10 monclient(hunting): handle_auth_done global_id 4171 payload 824 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 -- 10.20.193.112:0/3042398139 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb680119da0 msgr2=0x7fb6801ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb680119da0 0x7fb6801ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 -- 10.20.193.112:0/3042398139 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 msgr2=0x7fb6801b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb6801b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 monclient: found mon.noname-b 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.074+0000 7fb6858c9640 1 -- 10.20.193.112:0/3042398139 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6801abda0 con 0x7fb6801193d0 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.075+0000 7fb6858c9640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 0x7fb6801b1aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb67400e9b0 tx=0x7fb67400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.095 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.075+0000 7fb666ffd640 10 client.?.objecter ms_handle_connect 0x7fb6801193d0 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.075+0000 7fb666ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.076+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674017070 con 0x7fb6801193d0 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.076+0000 7fb666ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.076+0000 7fb666ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: dump: 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.096 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 5 monclient: authenticate success, global_id 4171 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _renew_subs 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6801ac030 con 0x7fb6801193d0 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 client.4171.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _renew_subs 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:50.097 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801b83c0 con 0x7fb6801193d0 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 1 librados: init done 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674005a90 con 0x7fb6801193d0 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674021060 con 0x7fb6801193d0 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb666ffd640 10 monclient: dump: 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.098 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb6848c7640 4 set_mon_vals no callback set 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:50.099 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.077+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb680114e30 con 0x7fb6801193d0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.087+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7fb674024060 con 0x7fb6801193d0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 0x7fb65007b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb6860ca640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 0x7fb65007b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb6860ca640 10 monclient: get_auth_request con 0x7fb650078c50 auth_method 0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 10 client.4171.objecter ms_handle_connect 0x7fb650078c50 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb6860ca640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 0x7fb65007b110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb67002ed70 tx=0x7fb670033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2105+0+0 (secure 0 0 0) 0x7fb67405dd50 con 0x7fb6801193d0 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 10 client.4171.objecter ms_dispatch2 0x7fb680000b90 osd_map(14..14 src has 1..14) 2026-02-24T18:33:50.101 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 3 client.4171.objecter handle_osd_map got epochs [14,14] > 0 2026-02-24T18:33:50.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 3 client.4171.objecter handle_osd_map decoding full epoch 14 2026-02-24T18:33:50.102 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.088+0000 7fb666ffd640 20 client.4171.objecter dump_active .. 0 homeless 2026-02-24T18:33:50.186 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.182+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb674061d10 con 0x7fb6801193d0 2026-02-24T18:33:50.186 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.183+0000 7fb666ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.186 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.183+0000 7fb666ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:50.276 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.272+0000 7fb688355640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "53546e48-a93a-4f5f-a114-9ea08210aba6", "id": 12}] 2026-02-24T18:33:50.276 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.272+0000 7fb688355640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "53546e48-a93a-4f5f-a114-9ea08210aba6", "id": 12}] 2026-02-24T18:33:50.276 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.272+0000 7fb688355640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:50.276 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.272+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "53546e48-a93a-4f5f-a114-9ea08210aba6", "id": 12} v 0) -- 0x7fb6801b85f0 con 0x7fb6801193d0 2026-02-24T18:33:50.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb666ffd640 1 -- 10.20.193.112:0/3042398139 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53546e48-a93a-4f5f-a114-9ea08210aba6", "id": 12}]=0 v15) ==== 113+0+3 (secure 0 0 0) 0x7fb674066010 con 0x7fb6801193d0 2026-02-24T18:33:50.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb666ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "53546e48-a93a-4f5f-a114-9ea08210aba6", "id": 12}] 2026-02-24T18:33:50.326 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb666ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:50.326 INFO:teuthology.orchestra.run.trial112.stdout:12 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 10 librados: watch_flush enter 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 10 librados: watch_flush exit 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 20 client.4171.objecter shutdown clearing up homeless session... 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 10 client.4171.objecter successfully canceled tick 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 msgr2=0x7fb65007b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 0x7fb65007b110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb67002ed70 tx=0x7fb670033000 comp rx=0 tx=0).stop 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 10 monclient: shutdown 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 msgr2=0x7fb6801b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 0x7fb6801b1aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb67400e9b0 tx=0x7fb67400ee80 comp rx=0 tx=0).stop 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 shutdown_connections 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb650078c50 0x7fb65007b110 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6801193d0 0x7fb6801b1aa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb680119da0 0x7fb6801ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 --2- 10.20.193.112:0/3042398139 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb680114a50 0x7fb6801b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 >> 10.20.193.112:0/3042398139 conn(0x7fb68006dd80 msgr2=0x7fb6801182c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 shutdown_connections 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.323+0000 7fb688355640 1 -- 10.20.193.112:0/3042398139 wait complete. 2026-02-24T18:33:50.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.324+0000 7fb688355640 1 librados: shutdown 2026-02-24T18:33:50.334 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63 13 2026-02-24T18:33:50.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: get_monmap_and_config 2026-02-24T18:33:50.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.397 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: monmap: 2026-02-24T18:33:50.397 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.397 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:50.395336+0000 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:50.395336+0000 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:50.398 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.409 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 Processor -- start 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 -- start start 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: init 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient: _add_conns ranks=[4,3,1,5,0,2] 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2ad811a950 0x7f2ad811ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-b con 0x7f2ad811a950 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2ad81151a0 0x7f2ad8115630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-a con 0x7f2ad81151a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2ad8115b70 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:50.410 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad8122d50 con 0x7f2ad8115b70 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad8123f30 con 0x7f2ad81151a0 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad81240b0 con 0x7f2ad811a950 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.394+0000 7f2adfc16640 10 monclient(hunting): authenticate will time out at 1372.040849s 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add98b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2ad811a950 0x7f2ad811ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2ad81151a0 0x7f2ad8115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2ade18c640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f2ad8115b70 0x7f2ad8121630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52176/0 (socket says 10.20.193.112:52176) 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2ade18c640 1 -- 10.20.193.112:0/3836852696 learned_addr learned my addr 10.20.193.112:0/3836852696 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:50.411 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): get_auth_request con 0x7f2ad81151a0 auth_method 0 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add98b640 10 monclient(hunting): get_auth_request con 0x7f2ad811a950 auth_method 0 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add98b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add98b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add98b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.395+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.412 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 10 monclient(hunting): handle_auth_done global_id 4191 payload 274 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 1 -- 10.20.193.112:0/3836852696 >> v2:10.20.193.32:3300/0 conn(0x7f2ad811a950 msgr2=0x7f2ad811ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 1 --2- 10.20.193.112:0/3836852696 >> v2:10.20.193.32:3300/0 conn(0x7f2ad811a950 0x7f2ad811ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 1 -- 10.20.193.112:0/3836852696 >> v1:10.20.193.32:6789/0 conn(0x7f2ad8115b70 legacy=0x7f2ad8121630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 1 monclient: found mon.noname-a 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.396+0000 7f2add18a640 1 -- 10.20.193.112:0/3836852696 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad8124230 con 0x7f2ad81151a0 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.397+0000 7f2add18a640 1 --2- 10.20.193.112:0/3836852696 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad8115630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ad806afd0 tx=0x7f2acc02ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839da515b70e0b43 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 1 -- 10.20.193.112:0/3836852696 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc03c070 con 0x7f2ad81151a0 2026-02-24T18:33:50.413 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: dump: 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.414 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 1 -- 10.20.193.112:0/3836852696 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc02fa60 con 0x7f2ad81151a0 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 5 monclient: authenticate success, global_id 4191 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 1 -- 10.20.193.112:0/3836852696 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc02fe30 con 0x7f2ad81151a0 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 4 set_mon_vals no callback set 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:50.415 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adc989640 10 monclient: dump: 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.416 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: shutdown 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- 10.20.193.112:0/3836852696 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 msgr2=0x7f2ad8115630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 --2- 10.20.193.112:0/3836852696 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad8115630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ad806afd0 tx=0x7f2acc02ed50 comp rx=0 tx=0).stop 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- 10.20.193.112:0/3836852696 shutdown_connections 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 --2- 10.20.193.112:0/3836852696 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad8115630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 --2- 10.20.193.112:0/3836852696 >> v2:10.20.193.32:3300/0 conn(0x7f2ad811a950 0x7f2ad811ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- 10.20.193.112:0/3836852696 >> 10.20.193.112:0/3836852696 conn(0x7f2ad806dfb0 msgr2=0x7f2ad806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- 10.20.193.112:0/3836852696 shutdown_connections 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- 10.20.193.112:0/3836852696 wait complete. 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: monmap: 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.417 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 librados: starting msgr at 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 librados: starting objecter 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 Processor -- start 2026-02-24T18:33:50.418 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 -- start start 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 librados: setting wanted keys 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 librados: calling monclient init 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: init 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-a con 0x7f2ad81151a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2ad8115b70 0x7f2ad81abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.399+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-c con 0x7f2ad8115b70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2ad811a950 0x7f2ad81b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): picked mon.noname-b con 0x7f2ad811a950 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.419 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81233b0 con 0x7f2ad81151a0 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8123230 con 0x7f2ad811a950 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8123530 con 0x7f2ad8115b70 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2adfc16640 10 monclient(hunting): authenticate will time out at 1372.046033s 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2ad8115b70 0x7f2ad81abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:59984/0 (socket says 10.20.193.112:59984) 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 1 -- 10.20.193.112:0/159524951 learned_addr learned my addr 10.20.193.112:0/159524951 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): get_auth_request con 0x7f2ad81151a0 auth_method 0 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.420 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): get_auth_request con 0x7f2ad8115b70 auth_method 0 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add98b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2ade18c640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2ad811a950 0x7f2ad81b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.400+0000 7f2add18a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.421 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): get_auth_request con 0x7f2ad811a950 auth_method 0 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2ade18c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 10 monclient(hunting): handle_auth_done global_id 4194 payload 824 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 -- 10.20.193.112:0/159524951 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2ad8115b70 msgr2=0x7f2ad81abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2ad8115b70 0x7f2ad81abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 -- 10.20.193.112:0/159524951 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2ad811a950 msgr2=0x7f2ad81b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2ad811a950 0x7f2ad81b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.422 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 monclient: found mon.noname-a 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.401+0000 7f2add98b640 1 -- 10.20.193.112:0/159524951 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad81b0970 con 0x7f2ad81151a0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.402+0000 7f2add98b640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800ebe0 tx=0x7f2ac800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.402+0000 7f2ac67fc640 10 client.?.objecter ms_handle_connect 0x7f2ad81151a0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.402+0000 7f2ac67fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8017070 con 0x7f2ad81151a0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 10 monclient: dump: 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.423 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2ac67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.424 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 5 monclient: authenticate success, global_id 4194 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 10 monclient: _renew_subs 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad81b0c60 con 0x7f2ad81151a0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 10 client.4194.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 10 monclient: _renew_subs 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81b84f0 con 0x7f2ad81151a0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.403+0000 7f2adfc16640 1 librados: init done 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2adfc16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2adfc16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2adfc16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad8115630 con 0x7f2ad81151a0 2026-02-24T18:33:50.425 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac800cc40 con 0x7f2ad81151a0 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8021060 con 0x7f2ad81151a0 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac7fff640 4 set_mon_vals no callback set 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.404+0000 7f2ac67fc640 10 monclient: dump: 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.426 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.427 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.427 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.427 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.512 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.509+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f2ac8024060 con 0x7f2ad81151a0 2026-02-24T18:33:50.513 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.510+0000 7f2ac67fc640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 0x7f2aa807b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.513 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.510+0000 7f2add18a640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 0x7f2aa807b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.514 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.510+0000 7f2add18a640 10 monclient: get_auth_request con 0x7f2aa8078c50 auth_method 0 2026-02-24T18:33:50.514 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.510+0000 7f2add18a640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 0x7f2aa807b110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2acc039000 tx=0x7f2acc033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.514 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.510+0000 7f2ac67fc640 10 client.4194.objecter ms_handle_connect 0x7f2aa8078c50 2026-02-24T18:33:50.519 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2220+0+0 (secure 0 0 0) 0x7f2ac805e160 con 0x7f2ad81151a0 2026-02-24T18:33:50.519 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 10 client.4194.objecter ms_dispatch2 0x7f2ad8000b90 osd_map(15..15 src has 1..15) 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 3 client.4194.objecter handle_osd_map got epochs [15,15] > 0 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 3 client.4194.objecter handle_osd_map decoding full epoch 15 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 20 client.4194.objecter dump_active .. 0 homeless 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2ac8062120 con 0x7f2ad81151a0 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.520 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.516+0000 7f2ac67fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:50.606 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.603+0000 7f2adfc16640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63", "id": 13}] 2026-02-24T18:33:50.607 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.603+0000 7f2adfc16640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63", "id": 13}] 2026-02-24T18:33:50.607 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.603+0000 7f2adfc16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.607 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.603+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63", "id": 13} v 0) -- 0x7f2ad81b8720 con 0x7f2ad81151a0 2026-02-24T18:33:50.623 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2ac67fc640 1 -- 10.20.193.112:0/159524951 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63", "id": 13}]=0 v16) ==== 113+0+3 (secure 0 0 0) 0x7f2ac8066420 con 0x7f2ad81151a0 2026-02-24T18:33:50.623 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2ac67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63", "id": 13}] 2026-02-24T18:33:50.623 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2ac67fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:50.623 INFO:teuthology.orchestra.run.trial112.stdout:13 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 10 librados: watch_flush enter 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 10 librados: watch_flush exit 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 20 client.4194.objecter shutdown clearing up homeless session... 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 10 client.4194.objecter successfully canceled tick 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 msgr2=0x7f2aa807b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 0x7f2aa807b110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2acc039000 tx=0x7f2acc033000 comp rx=0 tx=0).stop 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 10 monclient: shutdown 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.625 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 msgr2=0x7f2ad81ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800ebe0 tx=0x7f2ac800c520 comp rx=0 tx=0).stop 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 shutdown_connections 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2aa8078c50 0x7f2aa807b110 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2ad811a950 0x7f2ad81b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2ad8115b70 0x7f2ad81abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 --2- 10.20.193.112:0/159524951 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2ad81151a0 0x7f2ad81ab9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 >> 10.20.193.112:0/159524951 conn(0x7f2ad806dfb0 msgr2=0x7f2ad81187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.620+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 shutdown_connections 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.621+0000 7f2adfc16640 1 -- 10.20.193.112:0/159524951 wait complete. 2026-02-24T18:33:50.626 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.621+0000 7f2adfc16640 1 librados: shutdown 2026-02-24T18:33:50.631 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new 7a952355-2344-492b-aa45-dd403a3f5c2f 14 2026-02-24T18:33:50.695 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 10 monclient: get_monmap_and_config 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 10 monclient: monmap: 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:50.692829+0000 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:50.692829+0000 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.696 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 1 Processor -- start 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 1 -- start start 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.691+0000 7f3a259d3640 10 monclient: init 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.697 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient: _add_conns ranks=[0,1,3,4,2,5] 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3a20073930 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3a2011d960 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3a20121470 0x7f3a20123860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-a con 0x7f3a20121470 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a20123e60 con 0x7f3a20073930 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a20125080 con 0x7f3a2011d960 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3a20126280 con 0x7f3a20121470 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a259d3640 10 monclient(hunting): authenticate will time out at 1372.338195s 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a1e7fc640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f3a2011d960 0x7f3a2011fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52188/0 (socket says 10.20.193.112:52188) 2026-02-24T18:33:50.698 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.692+0000 7f3a1e7fc640 1 -- 10.20.193.112:0/3180829100 learned_addr learned my addr 10.20.193.112:0/3180829100 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 1 --2- 10.20.193.112:0/3180829100 >> v2:10.20.193.18:3300/0 conn(0x7f3a20121470 0x7f3a20123860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): get_auth_request con 0x7f3a20121470 auth_method 0 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.758 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.755+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_done global_id 4197 payload 274 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3180829100 >> v1:10.20.193.32:6789/0 conn(0x7f3a2011d960 legacy=0x7f3a2011fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3180829100 >> v1:10.20.193.18:6789/0 conn(0x7f3a20073930 legacy=0x7f3a20073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 1 monclient: found mon.noname-a 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.760 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.758+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3180829100 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a20126400 con 0x7f3a20121470 2026-02-24T18:33:50.762 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.759+0000 7f3a1f7fe640 1 --2- 10.20.193.112:0/3180829100 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a20123860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df5af80e1d41eac server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 1 -- 10.20.193.112:0/3180829100 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1003c070 con 0x7f3a20121470 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: dump: 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 1 -- 10.20.193.112:0/3180829100 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10037440 con 0x7f3a20121470 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.764 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 1 -- 10.20.193.112:0/3180829100 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10037810 con 0x7f3a20121470 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a1dffb640 10 monclient: dump: 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 5 monclient: authenticate success, global_id 4197 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:50.765 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 4 set_mon_vals no callback set 2026-02-24T18:33:50.766 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 10 monclient: shutdown 2026-02-24T18:33:50.766 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.766 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 -- 10.20.193.112:0/3180829100 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 msgr2=0x7f3a20123860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3180829100 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a20123860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009d90 tx=0x7f3a1002f1e0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 -- 10.20.193.112:0/3180829100 shutdown_connections 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3180829100 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a20123860 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 -- 10.20.193.112:0/3180829100 >> 10.20.193.112:0/3180829100 conn(0x7f3a2006f640 msgr2=0x7f3a20071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 -- 10.20.193.112:0/3180829100 shutdown_connections 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.761+0000 7f3a259d3640 1 -- 10.20.193.112:0/3180829100 wait complete. 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient: build_initial_monmap 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient: monmap: 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 librados: starting msgr at 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 librados: starting objecter 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 Processor -- start 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 -- start start 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 librados: setting wanted keys 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 librados: calling monclient init 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient: init 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:33:50.767 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a20073930 0x7f3a2011c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-b con 0x7f3a20073930 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a2011d960 0x7f3a2011c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-c con 0x7f3a2011d960 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): picked mon.noname-a con 0x7f3a20121470 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201244f0 con 0x7f3a20121470 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20124370 con 0x7f3a20073930 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20124670 con 0x7f3a2011d960 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a259d3640 10 monclient(hunting): authenticate will time out at 1372.408624s 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.762+0000 7f3a1f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.112:60004/0 (socket says 10.20.193.112:60004) 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3524405120 learned_addr learned my addr 10.20.193.112:0/3524405120 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): get_auth_request con 0x7f3a20121470 auth_method 0 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:50.768 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.763+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 10 monclient(hunting): handle_auth_done global_id 4203 payload 824 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3524405120 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a2011d960 msgr2=0x7f3a2011c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a2011d960 0x7f3a2011c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3524405120 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a20073930 msgr2=0x7f3a2011c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a20073930 0x7f3a2011c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 monclient: found mon.noname-a 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.764+0000 7f3a1f7fe640 1 -- 10.20.193.112:0/3524405120 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a201b1bd0 con 0x7f3a20121470 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.765+0000 7f3a1f7fe640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a1003a040 tx=0x7f3a10004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.765+0000 7f39f7fff640 10 client.?.objecter ms_handle_connect 0x7f3a20121470 2026-02-24T18:33:50.769 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.765+0000 7f39f7fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1003c050 con 0x7f3a20121470 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: dump: 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.770 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: _finish_auth 0 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10048070 con 0x7f3a20121470 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 5 monclient: authenticate success, global_id 4203 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _renew_subs 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a201b1e00 con 0x7f3a20121470 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10035d40 con 0x7f3a20121470 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f39f7fff640 10 monclient: dump: 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a1d7fa640 4 set_mon_vals no callback set 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 client.4203.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _renew_subs 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.771 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201b84a0 con 0x7f3a20121470 2026-02-24T18:33:50.772 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 1 librados: init done 2026-02-24T18:33:50.772 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.772 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.772 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.772 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.767+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20124b50 con 0x7f3a20121470 2026-02-24T18:33:50.781 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.778+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f3a1004d460 con 0x7f3a20121470 2026-02-24T18:33:50.781 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.779+0000 7f39f7fff640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 0x7f39f007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:50.782 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.779+0000 7f3a1effd640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 0x7f39f007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:50.782 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.779+0000 7f3a1effd640 10 monclient: get_auth_request con 0x7f39f0078dc0 auth_method 0 2026-02-24T18:33:50.782 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.779+0000 7f3a1effd640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 0x7f39f007b280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a14002f00 tx=0x7f3a14007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:50.782 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.779+0000 7f39f7fff640 10 client.4203.objecter ms_handle_connect 0x7f39f0078dc0 2026-02-24T18:33:50.783 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.780+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2335+0+0 (secure 0 0 0) 0x7f3a10084640 con 0x7f3a20121470 2026-02-24T18:33:50.783 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.780+0000 7f39f7fff640 10 client.4203.objecter ms_dispatch2 0x7f3a20000b90 osd_map(16..16 src has 1..16) 2026-02-24T18:33:50.783 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.780+0000 7f39f7fff640 3 client.4203.objecter handle_osd_map got epochs [16,16] > 0 2026-02-24T18:33:50.783 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.780+0000 7f39f7fff640 3 client.4203.objecter handle_osd_map decoding full epoch 16 2026-02-24T18:33:50.783 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.780+0000 7f39f7fff640 20 client.4203.objecter dump_active .. 0 homeless 2026-02-24T18:33:50.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.881+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3a10088d00 con 0x7f3a20121470 2026-02-24T18:33:50.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.881+0000 7f39f7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:50.884 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.881+0000 7f39f7fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:50.971 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.968+0000 7f3a259d3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "7a952355-2344-492b-aa45-dd403a3f5c2f", "id": 14}] 2026-02-24T18:33:50.971 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.968+0000 7f3a259d3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "7a952355-2344-492b-aa45-dd403a3f5c2f", "id": 14}] 2026-02-24T18:33:50.971 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.968+0000 7f3a259d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:33:50.972 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.968+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7a952355-2344-492b-aa45-dd403a3f5c2f", "id": 14} v 0) -- 0x7f3a2006fb70 con 0x7f3a20121470 2026-02-24T18:33:50.990 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.987+0000 7f39f7fff640 1 -- 10.20.193.112:0/3524405120 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a952355-2344-492b-aa45-dd403a3f5c2f", "id": 14}]=0 v17) ==== 113+0+3 (secure 0 0 0) 0x7f3a10088600 con 0x7f3a20121470 2026-02-24T18:33:50.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.987+0000 7f39f7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "7a952355-2344-492b-aa45-dd403a3f5c2f", "id": 14}] 2026-02-24T18:33:50.991 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.987+0000 7f39f7fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:50.991 INFO:teuthology.orchestra.run.trial112.stdout:14 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 10 librados: watch_flush enter 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 10 librados: watch_flush exit 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 20 client.4203.objecter shutdown clearing up homeless session... 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 10 client.4203.objecter successfully canceled tick 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 msgr2=0x7f39f007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 0x7f39f007b280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3a14002f00 tx=0x7f3a14007a70 comp rx=0 tx=0).stop 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 10 monclient: shutdown 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 msgr2=0x7f3a201b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a1003a040 tx=0x7f3a10004b40 comp rx=0 tx=0).stop 2026-02-24T18:33:50.993 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 shutdown_connections 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f39f0078dc0 0x7f39f007b280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a20121470 0x7f3a201b1310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a2011d960 0x7f3a2011c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 --2- 10.20.193.112:0/3524405120 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a20073930 0x7f3a2011c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.988+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 >> 10.20.193.112:0/3524405120 conn(0x7f3a2006f640 msgr2=0x7f3a20121850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.989+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 shutdown_connections 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.989+0000 7f3a259d3640 1 -- 10.20.193.112:0/3524405120 wait complete. 2026-02-24T18:33:50.994 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:50.989+0000 7f3a259d3640 1 librados: shutdown 2026-02-24T18:33:50.999 DEBUG:teuthology.orchestra.run.trial112:> sudo ceph --cluster ceph osd new a565a854-0fc5-46c1-ad7a-350d27bb20d2 15 2026-02-24T18:33:51.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: get_monmap_and_config 2026-02-24T18:33:51.064 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: build_initial_monmap 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: monmap: 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:33:51.061313+0000 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:33:51.061313+0000 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:51.065 INFO:teuthology.orchestra.run.trial112.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 Processor -- start 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 -- start start 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: init 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient: _add_conns ranks=[3,2,1,4,0,5] 2026-02-24T18:33:51.066 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1dac11a950 0x7f1dac11ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-a con 0x7f1dac11a950 addr v2:10.20.193.18:3300/0 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1dac1151a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1dac115d40 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac123e20 con 0x7f1dac115d40 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac125040 con 0x7f1dac1151a0 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac126240 con 0x7f1dac11a950 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.060+0000 7f1db47dd640 10 monclient(hunting): authenticate will time out at 1372.706692s 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.061+0000 7f1db2d53640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f1dac115d40 0x7f1dac122640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.112:52204/0 (socket says 10.20.193.112:52204) 2026-02-24T18:33:51.067 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.061+0000 7f1db2d53640 1 -- 10.20.193.112:0/3650410597 learned_addr learned my addr 10.20.193.112:0/3650410597 (peer_addr_for_me v1:10.20.193.112:0/0) 2026-02-24T18:33:51.079 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944440028 0 0) 0x7f1dac125040 con 0x7f1dac1151a0 2026-02-24T18:33:51.079 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _negotiate 2026-02-24T18:33:51.079 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.079 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.079 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): my global_id is 4202 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d94003620 con 0x7f1dac1151a0 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155327817 0 0) 0x7f1dac123e20 con 0x7f1dac115d40 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _negotiate 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 10 monclient(hunting): my global_id is 4186 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.062+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dac125040 con 0x7f1dac115d40 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3289901081 0 0) 0x7f1d94003620 con 0x7f1dac1151a0 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 >> v2:10.20.193.18:3300/0 conn(0x7f1dac11a950 msgr2=0x7f1dac11ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 --2- 10.20.193.112:0/3650410597 >> v2:10.20.193.18:3300/0 conn(0x7f1dac11a950 0x7f1dac11ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 >> v1:10.20.193.32:6789/0 conn(0x7f1dac115d40 legacy=0x7f1dac122640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _finish_hunting 0 2026-02-24T18:33:51.080 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1263c0 con 0x7f1dac1151a0 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _finish_auth 0 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _check_auth_tickets 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f1da0002f00 con 0x7f1dac1151a0 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: dump: 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.081 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 >> v1:10.20.193.110:6789/0 conn(0x7f1dac1151a0 legacy=0x7f1dac115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 0x7f1d940041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): picked mon.b con 0x7f1d94003dd0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d94004950 0x7f1d94004db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): picked mon.a con 0x7f1d94004950 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94009ab0 0x7f1d9400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.082 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): picked mon.c con 0x7f1d94009ab0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac126240 con 0x7f1d94004950 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9400c5a0 con 0x7f1d94003dd0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9400c7a0 con 0x7f1d94009ab0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db1550640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.064+0000 7f1db2552640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 0x7f1d940041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): get_auth_request con 0x7f1d94003dd0 auth_method 0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94009ab0 0x7f1d9400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.083 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): get_auth_request con 0x7f1d94009ab0 auth_method 0 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.065+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 10 monclient(hunting): handle_auth_done global_id 4202 payload 293 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 -- 10.20.193.112:0/3650410597 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94009ab0 msgr2=0x7f1d9400bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94009ab0 0x7f1d9400bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 -- 10.20.193.112:0/3650410597 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d94004950 msgr2=0x7f1d94004db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d94004950 0x7f1d94004db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 10 monclient: _finish_hunting 0 2026-02-24T18:33:51.084 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 monclient: found mon.b 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2d53640 1 -- 10.20.193.112:0/3650410597 reap_dead start 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 1 -- 10.20.193.112:0/3650410597 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d94003620 con 0x7f1d94003dd0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 10 monclient: _finish_auth 0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 10 monclient: _check_auth_tickets 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db2552640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db47dd640 5 monclient: authenticate success, global_id 4202 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.066+0000 7f1db47dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.067+0000 7f1db2552640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 0x7f1d940041d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009980 tx=0x7f1d9c02ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b847b677f71cf0e7 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03c070 con 0x7f1d94003dd0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: dump: 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.085 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9c02f790 con 0x7f1d94003dd0 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 1 -- 10.20.193.112:0/3650410597 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c009550 con 0x7f1d94003dd0 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db1550640 10 monclient: dump: 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.086 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 4 set_mon_vals no callback set 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 10 monclient: shutdown 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 -- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 msgr2=0x7f1d940041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 0x7f1d940041d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009980 tx=0x7f1d9c02ed30 comp rx=0 tx=0).stop 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 -- 10.20.193.112:0/3650410597 shutdown_connections 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 --2- 10.20.193.112:0/3650410597 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d94003dd0 0x7f1d940041d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 -- 10.20.193.112:0/3650410597 >> 10.20.193.112:0/3650410597 conn(0x7f1dac06dfb0 msgr2=0x7f1dac06e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 -- 10.20.193.112:0/3650410597 shutdown_connections 2026-02-24T18:33:51.087 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.068+0000 7f1db47dd640 1 -- 10.20.193.112:0/3650410597 wait complete. 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient: build_initial_monmap 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient: monmap: 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:epoch 0 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 0.000000 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:created 0.000000 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 librados: starting msgr at 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 librados: starting objecter 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 Processor -- start 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 -- start start 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 librados: setting wanted keys 2026-02-24T18:33:51.088 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 librados: calling monclient init 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient: init 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94003dd0 0x7f1dac1223a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-c con 0x7f1d94003dd0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1dac122970 0x7f1dac1b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-a con 0x7f1dac122970 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): picked mon.noname-b con 0x7f1dac1b1890 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac115630 con 0x7f1dac122970 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac11b000 con 0x7f1dac1b1890 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1157b0 con 0x7f1d94003dd0 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:51.089 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db47dd640 10 monclient(hunting): authenticate will time out at 1372.715527s 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2552640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94003dd0 0x7f1dac1223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.112:38854/0 (socket says 10.20.193.112:38854) 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 1 -- 10.20.193.112:0/430921454 learned_addr learned my addr 10.20.193.112:0/430921454 (peer_addr_for_me v2:10.20.193.112:0/0) 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 10 monclient(hunting): get_auth_request con 0x7f1dac1b1890 auth_method 0 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.069+0000 7f1db2d53640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): get_auth_request con 0x7f1d94003dd0 auth_method 0 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2d53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.090 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.070+0000 7f1db2552640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 10 monclient(hunting): handle_auth_done global_id 4189 payload 824 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 -- 10.20.193.112:0/430921454 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94003dd0 msgr2=0x7f1dac1223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94003dd0 0x7f1dac1223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 -- 10.20.193.112:0/430921454 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1dac122970 msgr2=0x7f1dac1b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1dac122970 0x7f1dac1b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 10 monclient: _finish_hunting 0 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 monclient: found mon.noname-b 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.071+0000 7f1db2d53640 1 -- 10.20.193.112:0/430921454 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1b8420 con 0x7f1dac1b1890 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.072+0000 7f1db2d53640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1dac06f260 tx=0x7f1da8007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.072+0000 7f1d92ffd640 10 client.?.objecter ms_handle_connect 0x7f1dac1b1890 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.072+0000 7f1d92ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da801a070 con 0x7f1dac1b1890 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:51.091 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: dump: 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: _finish_auth 0 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da800fe70 con 0x7f1dac1b1890 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8012550 con 0x7f1dac1b1890 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:51.092 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1d92ffd640 10 monclient: dump: 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:epoch 1 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:election_strategy: 1 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 5 monclient: authenticate success, global_id 4189 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db0d4f640 4 set_mon_vals no callback set 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _renew_subs 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1b86b0 con 0x7f1dac1b1890 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 client.4189.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _renew_subs 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1b8b80 con 0x7f1dac1b1890 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 1 librados: init done 2026-02-24T18:33:51.093 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.094 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.074+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac125210 con 0x7f1dac1b1890 2026-02-24T18:33:51.181 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.178+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f1da80126f0 con 0x7f1dac1b1890 2026-02-24T18:33:51.182 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.178+0000 7f1d92ffd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 0x7f1d7c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.182 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.179+0000 7f1db2552640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 0x7f1d7c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.182 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.179+0000 7f1db2552640 10 monclient: get_auth_request con 0x7f1d7c078dc0 auth_method 0 2026-02-24T18:33:51.182 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.179+0000 7f1db2552640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 0x7f1d7c07b280 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c039000 tx=0x7f1d9c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.182 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.179+0000 7f1d92ffd640 10 client.4189.objecter ms_handle_connect 0x7f1d7c078dc0 2026-02-24T18:33:51.192 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.189+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2450+0+0 (secure 0 0 0) 0x7f1da805de10 con 0x7f1dac1b1890 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.189+0000 7f1d92ffd640 10 client.4189.objecter ms_dispatch2 0x7f1dac000b90 osd_map(17..17 src has 1..17) 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.189+0000 7f1d92ffd640 3 client.4189.objecter handle_osd_map got epochs [17,17] > 0 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.189+0000 7f1d92ffd640 3 client.4189.objecter handle_osd_map decoding full epoch 17 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.189+0000 7f1d92ffd640 20 client.4189.objecter dump_active .. 0 homeless 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.191+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1da8024080 con 0x7f1dac1b1890 2026-02-24T18:33:51.193 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.191+0000 7f1d92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.194 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.191+0000 7f1d92ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:51.280 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.276+0000 7f1db47dd640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a565a854-0fc5-46c1-ad7a-350d27bb20d2", "id": 15}] 2026-02-24T18:33:51.280 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.276+0000 7f1db47dd640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a565a854-0fc5-46c1-ad7a-350d27bb20d2", "id": 15}] 2026-02-24T18:33:51.280 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.276+0000 7f1db47dd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:51.280 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.276+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a565a854-0fc5-46c1-ad7a-350d27bb20d2", "id": 15} v 0) -- 0x7f1dac11b770 con 0x7f1dac1b1890 2026-02-24T18:33:51.325 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.321+0000 7f1d92ffd640 1 -- 10.20.193.112:0/430921454 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a565a854-0fc5-46c1-ad7a-350d27bb20d2", "id": 15}]=0 v18) ==== 113+0+3 (secure 0 0 0) 0x7f1da8061dd0 con 0x7f1dac1b1890 2026-02-24T18:33:51.325 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.321+0000 7f1d92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a565a854-0fc5-46c1-ad7a-350d27bb20d2", "id": 15}] 2026-02-24T18:33:51.325 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.321+0000 7f1d92ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:51.326 INFO:teuthology.orchestra.run.trial112.stdout:15 2026-02-24T18:33:51.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 10 librados: watch_flush enter 2026-02-24T18:33:51.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 10 librados: watch_flush exit 2026-02-24T18:33:51.328 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 20 client.4189.objecter shutdown clearing up homeless session... 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 10 client.4189.objecter successfully canceled tick 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 msgr2=0x7f1d7c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 0x7f1d7c07b280 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c039000 tx=0x7f1d9c033000 comp rx=0 tx=0).stop 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 10 monclient: shutdown 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 msgr2=0x7f1dac1b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1dac06f260 tx=0x7f1da8007d00 comp rx=0 tx=0).stop 2026-02-24T18:33:51.329 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 shutdown_connections 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d7c078dc0 0x7f1d7c07b280 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1dac1b1890 0x7f1dac1b1d20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1dac122970 0x7f1dac1b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 --2- 10.20.193.112:0/430921454 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d94003dd0 0x7f1dac1223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 >> 10.20.193.112:0/430921454 conn(0x7f1dac06dfb0 msgr2=0x7f1dac113320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 shutdown_connections 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 -- 10.20.193.112:0/430921454 wait complete. 2026-02-24T18:33:51.330 INFO:teuthology.orchestra.run.trial112.stderr:2026-02-24T18:33:51.322+0000 7f1db47dd640 1 librados: shutdown 2026-02-24T18:33:51.333 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.334 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-24T18:33:51.334 DEBUG:teuthology.orchestra.run.trial018:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2026-02-24T18:33:51.336 INFO:tasks.ceph.osd.0:Started 2026-02-24T18:33:51.336 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.336 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-24T18:33:51.336 DEBUG:teuthology.orchestra.run.trial018:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2026-02-24T18:33:51.337 INFO:tasks.ceph.osd.4:Started 2026-02-24T18:33:51.337 DEBUG:tasks.ceph_manager:running osd.8 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.8.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.337 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-24T18:33:51.337 DEBUG:teuthology.orchestra.run.trial018:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 2026-02-24T18:33:51.338 INFO:tasks.ceph.osd.8:Started 2026-02-24T18:33:51.338 DEBUG:tasks.ceph_manager:running osd.12 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.12.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.338 INFO:tasks.ceph.osd.12:Restarting daemon 2026-02-24T18:33:51.338 DEBUG:teuthology.orchestra.run.trial018:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 2026-02-24T18:33:51.340 INFO:tasks.ceph.osd.12:Started 2026-02-24T18:33:51.340 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.340 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-24T18:33:51.340 DEBUG:teuthology.orchestra.run.trial032:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2026-02-24T18:33:51.341 INFO:tasks.ceph.osd.1:Started 2026-02-24T18:33:51.341 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.341 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-24T18:33:51.341 DEBUG:teuthology.orchestra.run.trial032:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2026-02-24T18:33:51.343 INFO:tasks.ceph.osd.5:Started 2026-02-24T18:33:51.343 DEBUG:tasks.ceph_manager:running osd.9 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.9.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.343 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-24T18:33:51.343 DEBUG:teuthology.orchestra.run.trial032:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 2026-02-24T18:33:51.346 INFO:tasks.ceph.osd.9:Started 2026-02-24T18:33:51.346 DEBUG:tasks.ceph_manager:running osd.13 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.13.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.346 INFO:tasks.ceph.osd.13:Restarting daemon 2026-02-24T18:33:51.347 DEBUG:teuthology.orchestra.run.trial032:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 2026-02-24T18:33:51.349 INFO:tasks.ceph.osd.13:Started 2026-02-24T18:33:51.349 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.350 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-24T18:33:51.350 DEBUG:teuthology.orchestra.run.trial110:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2026-02-24T18:33:51.352 INFO:tasks.ceph.osd.2:Started 2026-02-24T18:33:51.352 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.353 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-24T18:33:51.353 DEBUG:teuthology.orchestra.run.trial110:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2026-02-24T18:33:51.355 INFO:tasks.ceph.osd.6:Started 2026-02-24T18:33:51.355 DEBUG:tasks.ceph_manager:running osd.10 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.10.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.355 INFO:tasks.ceph.osd.10:Restarting daemon 2026-02-24T18:33:51.356 DEBUG:teuthology.orchestra.run.trial110:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 2026-02-24T18:33:51.358 INFO:tasks.ceph.osd.10:Started 2026-02-24T18:33:51.358 DEBUG:tasks.ceph_manager:running osd.14 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.14.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.358 INFO:tasks.ceph.osd.14:Restarting daemon 2026-02-24T18:33:51.359 DEBUG:teuthology.orchestra.run.trial110:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 2026-02-24T18:33:51.361 INFO:tasks.ceph.osd.14:Started 2026-02-24T18:33:51.361 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.361 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-24T18:33:51.362 DEBUG:teuthology.orchestra.run.trial112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2026-02-24T18:33:51.364 INFO:tasks.ceph.osd.3:Started 2026-02-24T18:33:51.364 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.364 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-24T18:33:51.365 DEBUG:teuthology.orchestra.run.trial112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2026-02-24T18:33:51.367 INFO:tasks.ceph.osd.7:Started 2026-02-24T18:33:51.367 DEBUG:tasks.ceph_manager:running osd.11 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.11.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.368 INFO:tasks.ceph.osd.11:Restarting daemon 2026-02-24T18:33:51.368 DEBUG:teuthology.orchestra.run.trial112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 2026-02-24T18:33:51.370 INFO:tasks.ceph.osd.11:Started 2026-02-24T18:33:51.370 DEBUG:tasks.ceph_manager:running osd.15 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.15.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:33:51.371 INFO:tasks.ceph.osd.15:Restarting daemon 2026-02-24T18:33:51.371 DEBUG:teuthology.orchestra.run.trial112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 2026-02-24T18:33:51.373 INFO:tasks.ceph.osd.15:Started 2026-02-24T18:33:51.374 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:33:51.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 10 monclient: get_monmap_and_config 2026-02-24T18:33:51.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 10 monclient: build_initial_monmap 2026-02-24T18:33:51.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 10 monclient: monmap: 2026-02-24T18:33:51.445 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:33:51.442975+0000 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:33:51.442975+0000 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 1 Processor -- start 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 1 -- start start 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.442+0000 7f8e979bc640 10 monclient: init 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient: _add_conns ranks=[5,2,1,3,0,4] 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f8e9006ff20 0x7f8e90070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-c con 0x7f8e9006ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8e900c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8e9015a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e900c7e90 con 0x7f8e9015a5e0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e900c8170 con 0x7f8e900c99a0 2026-02-24T18:33:51.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8e90159a40 con 0x7f8e9006ff20 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e979bc640 10 monclient(hunting): authenticate will time out at 1375.124123s 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e95f32640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f8e9015a5e0 0x7f8e9014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:46320/0 (socket says 10.20.193.18:46320) 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e95f32640 1 -- 10.20.193.18:0/3824272181 learned_addr learned my addr 10.20.193.18:0/3824272181 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.443+0000 7f8e95731640 1 --2- 10.20.193.18:0/3824272181 >> v2:10.20.193.110:3300/0 conn(0x7f8e9006ff20 0x7f8e90070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): get_auth_request con 0x7f8e9006ff20 auth_method 0 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.444+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 1 -- 10.20.193.18:0/3824272181 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514876235 0 0) 0x7f8e900c7e90 con 0x7f8e9015a5e0 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 10 monclient(hunting): _negotiate 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 10 monclient(hunting): my global_id is 4192 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e7ffff640 1 -- 10.20.193.18:0/3824272181 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e78003620 con 0x7f8e9015a5e0 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 10 monclient(hunting): handle_auth_done global_id 4208 payload 274 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 1 -- 10.20.193.18:0/3824272181 >> v1:10.20.193.110:6789/0 conn(0x7f8e900c99a0 legacy=0x7f8e90159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 1 -- 10.20.193.18:0/3824272181 >> v1:10.20.193.32:6789/0 conn(0x7f8e9015a5e0 legacy=0x7f8e9014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 10 monclient: _finish_hunting 0 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 1 monclient: found mon.noname-c 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.445+0000 7f8e95731640 1 -- 10.20.193.18:0/3824272181 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e900c8450 con 0x7f8e9006ff20 2026-02-24T18:33:51.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.446+0000 7f8e95731640 1 --2- 10.20.193.18:0/3824272181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e90070300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e800098d0 tx=0x7f8e8002ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a03d1aa196f063 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 1 -- 10.20.193.18:0/3824272181 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003c070 con 0x7f8e9006ff20 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: dump: 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.451 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: _finish_auth 0 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 1 -- 10.20.193.18:0/3824272181 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8002fb00 con 0x7f8e9006ff20 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 5 monclient: authenticate success, global_id 4208 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 1 -- 10.20.193.18:0/3824272181 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80037440 con 0x7f8e9006ff20 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 4 set_mon_vals no callback set 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e7ffff640 10 monclient: dump: 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.452 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 10 monclient: shutdown 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 1 -- 10.20.193.18:0/3824272181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 msgr2=0x7f8e90070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.449+0000 7f8e979bc640 1 --2- 10.20.193.18:0/3824272181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e90070300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e800098d0 tx=0x7f8e8002ee80 comp rx=0 tx=0).stop 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 -- 10.20.193.18:0/3824272181 shutdown_connections 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 --2- 10.20.193.18:0/3824272181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e90070300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 -- 10.20.193.18:0/3824272181 >> 10.20.193.18:0/3824272181 conn(0x7f8e900cd780 msgr2=0x7f8e900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 -- 10.20.193.18:0/3824272181 shutdown_connections 2026-02-24T18:33:51.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 -- 10.20.193.18:0/3824272181 wait complete. 2026-02-24T18:33:51.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 10 monclient: build_initial_monmap 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 10 monclient: monmap: 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 librados: starting msgr at 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 librados: starting objecter 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 Processor -- start 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 -- start start 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 librados: setting wanted keys 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 1 librados: calling monclient init 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.450+0000 7f8e979bc640 10 monclient: init 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-c con 0x7f8e9006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8e900c99a0 0x7f8e900c2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-a con 0x7f8e900c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:51.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8e9015a5e0 0x7f8e900c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): picked mon.noname-b con 0x7f8e9015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90150520 con 0x7f8e900c99a0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e900c8a10 con 0x7f8e9015a5e0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901506a0 con 0x7f8e9006ff20 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e979bc640 10 monclient(hunting): authenticate will time out at 1375.132279s 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95f32640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8e9015a5e0 0x7f8e900c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e94f30640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8e900c99a0 0x7f8e900c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:50596/0 (socket says 10.20.193.18:50596) 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 1 -- 10.20.193.18:0/422560740 learned_addr learned my addr 10.20.193.18:0/422560740 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 10 monclient(hunting): get_auth_request con 0x7f8e9006ff20 auth_method 0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95731640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95f32640 10 monclient(hunting): get_auth_request con 0x7f8e9015a5e0 auth_method 0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95f32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95f32640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.451+0000 7f8e95f32640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): get_auth_request con 0x7f8e900c99a0 auth_method 0 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:51.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95731640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95f32640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95f32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e95f32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.452+0000 7f8e94f30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 10 monclient(hunting): handle_auth_done global_id 4211 payload 824 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 -- 10.20.193.18:0/422560740 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8e9015a5e0 msgr2=0x7f8e900c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8e9015a5e0 0x7f8e900c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 -- 10.20.193.18:0/422560740 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8e900c99a0 msgr2=0x7f8e900c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8e900c99a0 0x7f8e900c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 10 monclient: _finish_hunting 0 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 monclient: found mon.noname-c 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.453+0000 7f8e95731640 1 -- 10.20.193.18:0/422560740 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e900be4c0 con 0x7f8e9006ff20 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.454+0000 7f8e95731640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009130 tx=0x7f8e800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.454+0000 7f8e7dffb640 10 client.?.objecter ms_handle_connect 0x7f8e9006ff20 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.454+0000 7f8e7dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003c040 con 0x7f8e9006ff20 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 10 monclient: dump: 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.458 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 10 monclient: _finish_auth 0 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e979bc640 5 monclient: authenticate success, global_id 4211 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e979bc640 10 monclient: _renew_subs 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e979bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.455+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e900be7a0 con 0x7f8e9006ff20 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 client.4211.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 monclient: _renew_subs 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e900bed60 con 0x7f8e9006ff20 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 1 librados: init done 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80047070 con 0x7f8e9006ff20 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80042650 con 0x7f8e9006ff20 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7dffb640 10 monclient: dump: 2026-02-24T18:33:51.459 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e7f7fe640 4 set_mon_vals no callback set 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.456+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e900befa0 con 0x7f8e9006ff20 2026-02-24T18:33:51.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.467+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f8e800427f0 con 0x7f8e9006ff20 2026-02-24T18:33:51.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.467+0000 7f8e7dffb640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 0x7f8e6007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:51.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e94f30640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 0x7f8e6007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:51.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e94f30640 10 monclient: get_auth_request con 0x7f8e60078d70 auth_method 0 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f8e800bd090 con 0x7f8e9006ff20 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 10 client.4211.objecter ms_dispatch2 0x7f8e90000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 3 client.4211.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 3 client.4211.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 20 client.4211.objecter dump_active .. 0 homeless 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e7dffb640 10 client.4211.objecter ms_handle_connect 0x7f8e60078d70 2026-02-24T18:33:51.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.468+0000 7f8e94f30640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 0x7f8e6007b230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e90158f00 tx=0x7f8e84007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:51.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.569+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8e800878c0 con 0x7f8e9006ff20 2026-02-24T18:33:51.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.569+0000 7f8e7dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:51.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.569+0000 7f8e7dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:51.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.656+0000 7f8e979bc640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:51.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.656+0000 7f8e979bc640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:51.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.656+0000 7f8e979bc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:33:51.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.656+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e90070300 con 0x7f8e9006ff20 2026-02-24T18:33:51.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.682+0000 7f8e7dffb640 1 -- 10.20.193.18:0/422560740 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f8e8008bbc0 con 0x7f8e9006ff20 2026-02-24T18:33:51.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.682+0000 7f8e7dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:51.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.682+0000 7f8e7dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:51.683 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:33:51.684 INFO:teuthology.orchestra.run.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.683+0000 7f8e979bc640 10 librados: watch_flush enter 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.683+0000 7f8e979bc640 10 librados: watch_flush exit 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.683+0000 7f8e979bc640 20 client.4211.objecter shutdown clearing up homeless session... 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.683+0000 7f8e979bc640 10 client.4211.objecter successfully canceled tick 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 msgr2=0x7f8e6007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 0x7f8e6007b230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e90158f00 tx=0x7f8e84007f30 comp rx=0 tx=0).stop 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 10 monclient: shutdown 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 msgr2=0x7f8e900c0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009130 tx=0x7f8e800047c0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 shutdown_connections 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8e60078d70 0x7f8e6007b230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8e9015a5e0 0x7f8e900c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8e900c99a0 0x7f8e900c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 --2- 10.20.193.18:0/422560740 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8e9006ff20 0x7f8e900c0460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:51.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 >> 10.20.193.18:0/422560740 conn(0x7f8e900cd780 msgr2=0x7f8e9006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:51.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 shutdown_connections 2026-02-24T18:33:51.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 -- 10.20.193.18:0/422560740 wait complete. 2026-02-24T18:33:51.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:33:51.684+0000 7f8e979bc640 1 librados: shutdown 2026-02-24T18:33:51.694 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-24T18:33:51.695 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-24T18:33:53.307 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:33:53.370 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 10 monclient: get_monmap_and_config 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 10 monclient: build_initial_monmap 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 10 monclient: monmap: 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:33:53.370085+0000 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:33:53.370085+0000 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:33:53.371 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:33:53.372 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:33:53.372 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:33:53.372 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:33:53.372 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.374 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 1 Processor -- start 2026-02-24T18:33:53.374 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 1 -- start start 2026-02-24T18:33:53.374 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.369+0000 7ffa89982640 10 monclient: init 2026-02-24T18:33:53.374 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:53.374 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient: _add_conns ranks=[0,1,4,5,2,3] 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffa84156000 addr v1:10.20.193.18:6789/0 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ffa8406e6c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ffa840cc200 0x7ffa8415f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-b con 0x7ffa840cc200 addr v2:10.20.193.32:3300/0 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840cc9a0 con 0x7ffa84156000 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840ccc80 con 0x7ffa8406e6c0 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffa840cd890 con 0x7ffa840cc200 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa89982640 10 monclient(hunting): authenticate will time out at 1377.051298s 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa7bfff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7ffa8406e6c0 0x7ffa8415d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:46332/0 (socket says 10.20.193.18:46332) 2026-02-24T18:33:53.375 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.370+0000 7ffa7bfff640 1 -- 10.20.193.18:0/4290076391 learned_addr learned my addr 10.20.193.18:0/4290076391 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa89181640 1 --2- 10.20.193.18:0/4290076391 >> v2:10.20.193.32:3300/0 conn(0x7ffa840cc200 0x7ffa8415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1396921198 0 0) 0x7ffa840cc9a0 con 0x7ffa84156000 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): my global_id is 4209 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa70003620 con 0x7ffa84156000 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977225201 0 0) 0x7ffa840ccc80 con 0x7ffa8406e6c0 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.376 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 10 monclient(hunting): my global_id is 4198 2026-02-24T18:33:53.377 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa840cc9a0 con 0x7ffa8406e6c0 2026-02-24T18:33:53.377 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa89181640 10 monclient(hunting): get_auth_request con 0x7ffa840cc200 auth_method 0 2026-02-24T18:33:53.377 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa89181640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.377 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa89181640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.377 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.372+0000 7ffa89181640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2999297972 0 0) 0x7ffa70003620 con 0x7ffa84156000 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 >> v2:10.20.193.32:3300/0 conn(0x7ffa840cc200 msgr2=0x7ffa8415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 --2- 10.20.193.18:0/4290076391 >> v2:10.20.193.32:3300/0 conn(0x7ffa840cc200 0x7ffa8415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 >> v1:10.20.193.32:6789/0 conn(0x7ffa8406e6c0 legacy=0x7ffa8415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:33:53.386 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa840ccf60 con 0x7ffa84156000 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7ffa74002f00 con 0x7ffa84156000 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: dump: 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:53.387 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 >> v1:10.20.193.18:6789/0 conn(0x7ffa84156000 legacy=0x7ffa840cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004de0 0x7ffa700051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): picked mon.a con 0x7ffa70004de0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa700038a0 0x7ffa70003d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.388 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): picked mon.c con 0x7ffa700038a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 0x7ffa7000bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): picked mon.b con 0x7ffa70004420 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cd890 con 0x7ffa70004de0 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7000c3c0 con 0x7ffa70004420 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7000c5c0 con 0x7ffa700038a0 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.374+0000 7ffa7b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa700038a0 0x7ffa70003d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.389 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa88980640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004de0 0x7ffa700051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request con 0x7ffa700038a0 auth_method 0 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa88980640 10 monclient(hunting): get_auth_request con 0x7ffa70004de0 auth_method 0 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa88980640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa88980640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa88980640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.375+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 0x7ffa7000bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): get_auth_request con 0x7ffa70004420 auth_method 0 2026-02-24T18:33:53.390 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.376+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 10 monclient(hunting): handle_auth_done global_id 4209 payload 293 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 -- 10.20.193.18:0/4290076391 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa700038a0 msgr2=0x7ffa70003d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa700038a0 0x7ffa70003d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.391 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 -- 10.20.193.18:0/4290076391 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004de0 msgr2=0x7ffa700051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004de0 0x7ffa700051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 10 monclient: _finish_hunting 0 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 monclient: found mon.b 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 -- 10.20.193.18:0/4290076391 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70003620 con 0x7ffa70004420 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 10 monclient: _finish_auth 0 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 10 monclient: _check_auth_tickets 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89982640 5 monclient: authenticate success, global_id 4209 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89982640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.377+0000 7ffa89181640 1 -- 10.20.193.18:0/4290076391 reap_dead start 2026-02-24T18:33:53.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.378+0000 7ffa89181640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 0x7ffa7000bae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa80007360 tx=0x7ffa8002f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800b5532cfe2cda server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80009be0 con 0x7ffa70004420 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: dump: 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:53.393 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa80003cd0 con 0x7ffa70004420 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 10 monclient: get_monmap_and_config success 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 1 -- 10.20.193.18:0/4290076391 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80004260 con 0x7ffa70004420 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 4 set_mon_vals no callback set 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa7b7fe640 10 monclient: dump: 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.394 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 10 monclient: shutdown 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 -- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 msgr2=0x7ffa7000bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 0x7ffa7000bae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa80007360 tx=0x7ffa8002f7a0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 -- 10.20.193.18:0/4290076391 shutdown_connections 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 --2- 10.20.193.18:0/4290076391 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa70004420 0x7ffa7000bae0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 -- 10.20.193.18:0/4290076391 >> 10.20.193.18:0/4290076391 conn(0x7ffa8406d9e0 msgr2=0x7ffa8406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 -- 10.20.193.18:0/4290076391 shutdown_connections 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.380+0000 7ffa89982640 1 -- 10.20.193.18:0/4290076391 wait complete. 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 10 monclient: build_initial_monmap 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 10 monclient: monmap: 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:33:53.396 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 librados: starting msgr at 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 librados: starting objecter 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 Processor -- start 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 -- start start 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 librados: setting wanted keys 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 1 librados: calling monclient init 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.381+0000 7ffa89982640 10 monclient: init 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient: _reopen_session rank -1 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:33:53.397 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004420 0x7ffa841f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-a con 0x7ffa70004420 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa841f5220 0x7ffa841f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-c con 0x7ffa841f5220 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): picked mon.noname-b con 0x7ffa84152450 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cc640 con 0x7ffa70004420 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cd520 con 0x7ffa84152450 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): start opening mon connection 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841563e0 con 0x7ffa841f5220 2026-02-24T18:33:53.398 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): _renew_subs 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89982640 10 monclient(hunting): authenticate will time out at 1377.063082s 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004420 0x7ffa841f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa7bfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa841f5220 0x7ffa841f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:39026/0 (socket says 10.20.193.18:39026) 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 1 -- 10.20.193.18:0/1230884363 learned_addr learned my addr 10.20.193.18:0/1230884363 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): get_auth_request con 0x7ffa84152450 auth_method 0 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): get_auth_request con 0x7ffa70004420 auth_method 0 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.399 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa89181640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request con 0x7ffa841f5220 auth_method 0 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.382+0000 7ffa88980640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.383+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.383+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:33:53.400 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.383+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 10 monclient(hunting): handle_auth_done global_id 4204 payload 824 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 -- 10.20.193.18:0/1230884363 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa841f5220 msgr2=0x7ffa841f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa841f5220 0x7ffa841f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 -- 10.20.193.18:0/1230884363 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004420 msgr2=0x7ffa841f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004420 0x7ffa841f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 10 monclient: _finish_hunting 0 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 monclient: found mon.noname-b 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.401 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.384+0000 7ffa89181640 1 -- 10.20.193.18:0/1230884363 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa841f5880 con 0x7ffa84152450 2026-02-24T18:33:53.404 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.385+0000 7ffa89181640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002fcb0 tx=0x7ffa80004bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:53.404 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.385+0000 7ffa797fa640 10 client.?.objecter ms_handle_connect 0x7ffa84152450 2026-02-24T18:33:53.404 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.385+0000 7ffa797fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:33:53.404 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8003b040 con 0x7ffa84152450 2026-02-24T18:33:53.404 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: dump: 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.405 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: _finish_auth 0 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: _check_auth_tickets 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa80046070 con 0x7ffa84152450 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80009be0 con 0x7ffa84152450 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa7affd640 4 set_mon_vals no callback set 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa797fa640 10 monclient: dump: 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:33:53.406 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 5 monclient: authenticate success, global_id 4204 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 10 monclient: _renew_subs 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa841f5b60 con 0x7ffa84152450 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 10 client.4204.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 10 monclient: _renew_subs 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa841f6120 con 0x7ffa84152450 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.386+0000 7ffa89982640 1 librados: init done 2026-02-24T18:33:53.407 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.387+0000 7ffa89982640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.387+0000 7ffa89982640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.387+0000 7ffa89982640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.387+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8406a1d0 con 0x7ffa84152450 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.399+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7ffa8004b460 con 0x7ffa84152450 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.400+0000 7ffa797fa640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 0x7ffa5407b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.400+0000 7ffa88980640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 0x7ffa5407b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.400+0000 7ffa88980640 10 monclient: get_auth_request con 0x7ffa54078da0 auth_method 0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.401+0000 7ffa88980640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 0x7ffa5407b260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa740027e0 tx=0x7ffa74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.401+0000 7ffa797fa640 10 client.4204.objecter ms_handle_connect 0x7ffa54078da0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.402+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7ffa800837b0 con 0x7ffa84152450 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.402+0000 7ffa797fa640 10 client.4204.objecter ms_dispatch2 0x7ffa84000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.402+0000 7ffa797fa640 3 client.4204.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:33:53.408 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.402+0000 7ffa797fa640 3 client.4204.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:33:53.409 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.402+0000 7ffa797fa640 20 client.4204.objecter dump_active .. 0 homeless 2026-02-24T18:33:53.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.503+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffa8406a1d0 con 0x7ffa84152450 2026-02-24T18:33:53.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.503+0000 7ffa797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:33:53.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.503+0000 7ffa797fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:33:53.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.595+0000 7ffa89982640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:53.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.595+0000 7ffa89982640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:53.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.595+0000 7ffa89982640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:33:53.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.595+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa840cea40 con 0x7ffa84152450 2026-02-24T18:33:53.623 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.621+0000 7ffa797fa640 1 -- 10.20.193.18:0/1230884363 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7ffa80049360 con 0x7ffa84152450 2026-02-24T18:33:53.623 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.621+0000 7ffa797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:33:53.623 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.621+0000 7ffa797fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:33:53.623 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:33:53.623 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:33:53.626 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.623+0000 7ffa89982640 10 librados: watch_flush enter 2026-02-24T18:33:53.626 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.623+0000 7ffa89982640 10 librados: watch_flush exit 2026-02-24T18:33:53.626 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 20 client.4204.objecter shutdown clearing up homeless session... 2026-02-24T18:33:53.626 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 10 client.4204.objecter successfully canceled tick 2026-02-24T18:33:53.626 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 msgr2=0x7ffa5407b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 0x7ffa5407b260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa740027e0 tx=0x7ffa74009040 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 10 monclient: shutdown 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 msgr2=0x7ffa84152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002fcb0 tx=0x7ffa80004bb0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 shutdown_connections 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffa54078da0 0x7ffa5407b260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffa84152450 0x7ffa84152900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffa841f5220 0x7ffa841f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 --2- 10.20.193.18:0/1230884363 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffa70004420 0x7ffa841f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 >> 10.20.193.18:0/1230884363 conn(0x7ffa8406d9e0 msgr2=0x7ffa84151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:33:53.627 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 shutdown_connections 2026-02-24T18:33:53.628 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 -- 10.20.193.18:0/1230884363 wait complete. 2026-02-24T18:33:53.628 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:33:53.624+0000 7ffa89982640 1 librados: shutdown 2026-02-24T18:33:53.634 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:01.000 INFO:tasks.ceph.osd.6.trial110.stderr:--00:00:00:09.600 45415-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.077 INFO:tasks.ceph.osd.1.trial032.stderr:--00:00:00:09.690 44816-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.078 INFO:tasks.ceph.osd.9.trial032.stderr:--00:00:00:09.687 44818-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.083 INFO:tasks.ceph.osd.5.trial032.stderr:--00:00:00:09.694 44817-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.128 INFO:tasks.ceph.osd.14.trial110.stderr:--00:00:00:09.722 45417-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.129 INFO:tasks.ceph.osd.10.trial110.stderr:--00:00:00:09.723 45416-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.129 INFO:tasks.ceph.osd.2.trial110.stderr:--00:00:00:09.731 45414-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.171 INFO:tasks.ceph.osd.0.trial018.stderr:--00:00:00:09.787 45706-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.171 INFO:tasks.ceph.osd.12.trial018.stderr:--00:00:00:09.783 45713-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.174 INFO:tasks.ceph.osd.8.trial018.stderr:--00:00:00:09.788 45710-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.179 INFO:tasks.ceph.osd.4.trial018.stderr:--00:00:00:09.795 45705-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.203 INFO:tasks.ceph.osd.7.trial112.stderr:--00:00:00:09.790 45529-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.205 INFO:tasks.ceph.osd.11.trial112.stderr:--00:00:00:09.789 45530-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.207 INFO:tasks.ceph.osd.15.trial112.stderr:--00:00:00:09.790 45531-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.208 INFO:tasks.ceph.osd.3.trial112.stderr:--00:00:00:09.794 45528-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.244 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:01.259 INFO:tasks.ceph.osd.13.trial032.stderr:--00:00:00:09.863 44819-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:34:01.308 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.306+0000 7f7812ab0640 10 monclient: get_monmap_and_config 2026-02-24T18:34:01.308 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.306+0000 7f7812ab0640 10 monclient: build_initial_monmap 2026-02-24T18:34:01.308 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 10 monclient: monmap: 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:01.307859+0000 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:01.307859+0000 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:01.309 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:01.310 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.312 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 1 Processor -- start 2026-02-24T18:34:01.312 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 1 -- start start 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 10 monclient: init 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.307+0000 7f7812ab0640 10 monclient: _add_conns ranks=[0,2,5,3,1,4] 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f780c152a40 addr v1:10.20.193.18:6789/0 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f780c152e20 addr v1:10.20.193.110:6789/0 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f780c06e450 0x7f780c0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-c con 0x7f780c06e450 addr v2:10.20.193.110:3300/0 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780c0c6d70 con 0x7f780c152a40 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780c0c7050 con 0x7f780c152e20 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f780c0c8550 con 0x7f780c06e450 2026-02-24T18:34:01.313 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7812ab0640 10 monclient(hunting): authenticate will time out at 1384.989092s 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7810825640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f780c152a40 0x7f780c0c7e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58594/0 (socket says 10.20.193.18:58594) 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.308+0000 7f7810825640 1 -- 10.20.193.18:0/3890881647 learned_addr learned my addr 10.20.193.18:0/3890881647 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.309+0000 7f7811026640 1 --2- 10.20.193.18:0/3890881647 >> v2:10.20.193.110:3300/0 conn(0x7f780c06e450 0x7f780c0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.309+0000 7f7811026640 10 monclient(hunting): get_auth_request con 0x7f780c06e450 auth_method 0 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.309+0000 7f7811026640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.309+0000 7f7811026640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.309+0000 7f7811026640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2287753680 0 0) 0x7f780c0c6d70 con 0x7f780c152a40 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 10 monclient(hunting): _negotiate 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 10 monclient(hunting): my global_id is 4215 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.310+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77f80058b0 con 0x7f780c152a40 2026-02-24T18:34:01.314 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991688486 0 0) 0x7f780c0c7050 con 0x7f780c152e20 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 10 monclient(hunting): _negotiate 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 10 monclient(hunting): my global_id is 4217 2026-02-24T18:34:01.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.311+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f780c0c6d70 con 0x7f780c152e20 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3699145642 0 0) 0x7f77f80058b0 con 0x7f780c152a40 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 >> v2:10.20.193.110:3300/0 conn(0x7f780c06e450 msgr2=0x7f780c0c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 --2- 10.20.193.18:0/3890881647 >> v2:10.20.193.110:3300/0 conn(0x7f780c06e450 0x7f780c0c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 >> v1:10.20.193.110:6789/0 conn(0x7f780c152e20 legacy=0x7f780c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _finish_hunting 0 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:01.317 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c0c7330 con 0x7f780c152a40 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _finish_auth 0 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _check_auth_tickets 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f77fc002f00 con 0x7f780c152a40 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: dump: 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:01.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 >> v1:10.20.193.18:6789/0 conn(0x7f780c152a40 legacy=0x7f780c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006060 0x7f77f8006460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): picked mon.a con 0x7f77f8006060 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 0x7f77f8007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): picked mon.b con 0x7f77f8006b90 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f77f8007720 0x7f77f800dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): picked mon.c con 0x7f77f8007720 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f780c0c8550 con 0x7f77f8006060 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f77f800e610 con 0x7f77f8006b90 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77f800e810 con 0x7f77f8007720 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f78037fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.312+0000 7f7803fff640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 0x7f77f8007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): get_auth_request con 0x7f77f8006b90 auth_method 0 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.313+0000 7f7810825640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006060 0x7f77f8006460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): get_auth_request con 0x7f77f8006060 auth_method 0 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7811026640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f77f8007720 0x7f77f800dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 10 monclient(hunting): handle_auth_done global_id 4215 payload 293 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 -- 10.20.193.18:0/3890881647 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f77f8007720 msgr2=0x7f77f800dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f77f8007720 0x7f77f800dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 -- 10.20.193.18:0/3890881647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006060 msgr2=0x7f77f8006460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006060 0x7f77f8006460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 10 monclient: _finish_hunting 0 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 monclient: found mon.b 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:01.320 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7811026640 1 -- 10.20.193.18:0/3890881647 reap_dead start 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 1 -- 10.20.193.18:0/3890881647 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f80058b0 con 0x7f77f8006b90 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 10 monclient: _finish_auth 0 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 10 monclient: _check_auth_tickets 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7803fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7812ab0640 5 monclient: authenticate success, global_id 4215 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.314+0000 7f7812ab0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:01.321 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.316+0000 7f7803fff640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 0x7f77f8007010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77f4003210 tx=0x7f77f402eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5088bd9d408fc13c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.317+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4002e30 con 0x7f77f8006b90 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.317+0000 7f78037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.317+0000 7f78037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.317+0000 7f78037fe640 10 monclient: dump: 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.324 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.317+0000 7f7812ab0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f40052f0 con 0x7f77f8006b90 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 1 -- 10.20.193.18:0/3890881647 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f40068d0 con 0x7f77f8006b90 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f78037fe640 10 monclient: dump: 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 4 set_mon_vals no callback set 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 10 monclient: shutdown 2026-02-24T18:34:01.325 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 -- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 msgr2=0x7f77f8007010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 0x7f77f8007010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77f4003210 tx=0x7f77f402eeb0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 -- 10.20.193.18:0/3890881647 shutdown_connections 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 --2- 10.20.193.18:0/3890881647 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f77f8006b90 0x7f77f8007010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 -- 10.20.193.18:0/3890881647 >> 10.20.193.18:0/3890881647 conn(0x7f780c0c9f40 msgr2=0x7f780c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 -- 10.20.193.18:0/3890881647 shutdown_connections 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 1 -- 10.20.193.18:0/3890881647 wait complete. 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 10 monclient: build_initial_monmap 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.318+0000 7f7812ab0640 10 monclient: monmap: 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 librados: starting msgr at 2026-02-24T18:34:01.326 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 librados: starting objecter 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 Processor -- start 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 -- start start 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 librados: setting wanted keys 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 librados: calling monclient init 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient: init 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-a con 0x7f77f8006b90 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f780c14b580 0x7f780c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-b con 0x7f780c14b580 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f780c1f3d80 0x7f780c1f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): picked mon.noname-c con 0x7f780c1f3d80 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f780c06d820 con 0x7f77f8006b90 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f780c06d5d0 con 0x7f780c14b580 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f780c0c6960 con 0x7f780c1f3d80 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7812ab0640 10 monclient(hunting): authenticate will time out at 1385.000649s 2026-02-24T18:34:01.327 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.319+0000 7f7810825640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f780c1f3d80 0x7f780c1f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49908/0 (socket says 10.20.193.18:49908) 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 1 -- 10.20.193.18:0/938262828 learned_addr learned my addr 10.20.193.18:0/938262828 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): get_auth_request con 0x7f77f8006b90 auth_method 0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): get_auth_request con 0x7f780c1f3d80 auth_method 0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7803fff640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f780c14b580 0x7f780c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7810825640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7811026640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7803fff640 10 monclient(hunting): get_auth_request con 0x7f780c14b580 auth_method 0 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7803fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:01.328 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7803fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.320+0000 7f7803fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.321+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.321+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.321+0000 7f7803fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 10 monclient(hunting): handle_auth_done global_id 4218 payload 824 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 -- 10.20.193.18:0/938262828 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f780c1f3d80 msgr2=0x7f780c1f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f780c1f3d80 0x7f780c1f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 -- 10.20.193.18:0/938262828 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f780c14b580 msgr2=0x7f780c14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f780c14b580 0x7f780c14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 10 monclient: _finish_hunting 0 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 monclient: found mon.noname-a 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:34:01.329 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.322+0000 7f7810825640 1 -- 10.20.193.18:0/938262828 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c072b00 con 0x7f77f8006b90 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.323+0000 7f7810825640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0072c0 tx=0x7f77fc007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.323+0000 7f78017fa640 10 client.?.objecter ms_handle_connect 0x7f77f8006b90 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.323+0000 7f78017fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc002f00 con 0x7f77f8006b90 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:34:01.330 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: dump: 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: _finish_auth 0 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: _check_auth_tickets 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc018070 con 0x7f77f8006b90 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 5 monclient: authenticate success, global_id 4218 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _renew_subs 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1f4870 con 0x7f77f8006b90 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00fb20 con 0x7f77f8006b90 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:01.331 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f78017fa640 10 monclient: dump: 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7802ffd640 4 set_mon_vals no callback set 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 client.4218.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _renew_subs 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c072dc0 con 0x7f77f8006b90 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 1 librados: init done 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:01.332 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.325+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c15cc40 con 0x7f77f8006b90 2026-02-24T18:34:01.339 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.338+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f77fc00fcc0 con 0x7f77f8006b90 2026-02-24T18:34:01.339 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.338+0000 7f78017fa640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 0x7f77dc07b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:01.340 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.338+0000 7f7803fff640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 0x7f77dc07b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:01.340 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.339+0000 7f7803fff640 10 monclient: get_auth_request con 0x7f77dc078da0 auth_method 0 2026-02-24T18:34:01.340 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.339+0000 7f78017fa640 10 client.4218.objecter ms_handle_connect 0x7f77dc078da0 2026-02-24T18:34:01.340 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.339+0000 7f7803fff640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 0x7f77dc07b260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77f4036000 tx=0x7f77f403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:01.447 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f77fc05e390 con 0x7f77f8006b90 2026-02-24T18:34:01.447 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 10 client.4218.objecter ms_dispatch2 0x7f780c000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:01.447 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 3 client.4218.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:01.447 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 3 client.4218.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:01.448 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 20 client.4218.objecter dump_active .. 0 homeless 2026-02-24T18:34:01.448 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f77fc062350 con 0x7f77f8006b90 2026-02-24T18:34:01.448 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:01.448 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.445+0000 7f78017fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:01.538 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.536+0000 7f7812ab0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:01.538 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.536+0000 7f7812ab0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:01.539 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.536+0000 7f7812ab0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:01.539 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.536+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f780c156350 con 0x7f77f8006b90 2026-02-24T18:34:01.565 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.563+0000 7f78017fa640 1 -- 10.20.193.18:0/938262828 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f77fc066650 con 0x7f77f8006b90 2026-02-24T18:34:01.565 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.563+0000 7f78017fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:01.565 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.563+0000 7f78017fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:01.566 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:01.566 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:01.569 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 10 librados: watch_flush enter 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 10 librados: watch_flush exit 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 20 client.4218.objecter shutdown clearing up homeless session... 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 10 client.4218.objecter successfully canceled tick 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 msgr2=0x7f77dc07b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 0x7f77dc07b260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77f4036000 tx=0x7f77f403b040 comp rx=0 tx=0).stop 2026-02-24T18:34:01.570 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 10 monclient: shutdown 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 msgr2=0x7f780c1f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0072c0 tx=0x7f77fc007790 comp rx=0 tx=0).stop 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 shutdown_connections 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f77dc078da0 0x7f77dc07b260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f780c14b580 0x7f780c14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.571 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f780c1f3d80 0x7f780c1f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.572 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 --2- 10.20.193.18:0/938262828 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f77f8006b90 0x7f780c1f7c90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:01.572 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 >> 10.20.193.18:0/938262828 conn(0x7f780c0c9f40 msgr2=0x7f780c0c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:01.572 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 shutdown_connections 2026-02-24T18:34:01.572 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 -- 10.20.193.18:0/938262828 wait complete. 2026-02-24T18:34:01.572 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:01.565+0000 7f7812ab0640 1 librados: shutdown 2026-02-24T18:34:01.575 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:05.503 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.103 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.605 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:14.218 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.613 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:14.207 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.650 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:14.262 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.657 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:14.260 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.658 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:14.253 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.673 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:14.283 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.736 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:14.322 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.738 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:14.355 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.743 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:14.325 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.767 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:14.353 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.767 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:14.352 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.775 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:14.389 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.776 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:14.388 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.776 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:14.392 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.787 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:14.392 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.794 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.394 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.821 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.420 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.843 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.443 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.891 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:14.503 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.896 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:14.490 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.915 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:14.527 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.944 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:14.539 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.949 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:14.562 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.952 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:14.564 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.952 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:14.554 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.962 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:14.572 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.974 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:14.568 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.987 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:14.598 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:05.995 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:14.590 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.020 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:14.637 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.021 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:14.607 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.037 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:14.620 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.050 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:14.666 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.057 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:14.643 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.061 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:14.645 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.061 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:14.675 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.062 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:14.645 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.068 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:14.681 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.069 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:14.685 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.076 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:14.681 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.079 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:14.666 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.082 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:14.696 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.083 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:14.667 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.090 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:14.706 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.090 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:14.702 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.107 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.706 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.164 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:14.758 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.183 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:14.777 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.213 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:14.825 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.216 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:14.818 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.239 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:14.848 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.253 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:14.864 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.259 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:14.854 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.284 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:14.870 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.314 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:14.930 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.326 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:14.909 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.342 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:14.947 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.351 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:14.938 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.355 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:14.940 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.356 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:14.970 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.360 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:14.972 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.364 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:14.980 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.373 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.973 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.388 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:14.988 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.447 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:15.041 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.479 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:15.081 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.479 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:15.092 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.495 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:15.107 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.523 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:15.132 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.532 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:15.126 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.535 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:15.146 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.547 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:15.142 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.548 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:15.134 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.565 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:15.177 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.581 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:15.193 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.586 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:15.202 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.590 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:15.173 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.612 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:15.216 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.622 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:15.236 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.622 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:15.206 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.622 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:15.209 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.634 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:15.250 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.634 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:15.246 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.716 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:15.310 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.731 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:15.326 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.758 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:15.360 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.784 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:15.386 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.795 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:15.404 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.799 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:15.401 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.812 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:15.398 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.852 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:15.468 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.855 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:15.438 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.868 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:15.484 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.870 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:15.453 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.882 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:15.486 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.894 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:15.508 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.895 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:15.479 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.896 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:15.482 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.902 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:15.518 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.906 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:15.518 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.908 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:15.522 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.910 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:15.494 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.912 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:15.498 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.918 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:15.535 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:06.919 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:34:06.915+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:06.922 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:15.534 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.023 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:34:07.017+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.065 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:15.674 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.078 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:15.664 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.080 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:15.690 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.084 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:34:07.080+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.093 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:15.680 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.119 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:34:07.112+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.158 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:15.762 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.173 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:15.778 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:07.279 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:34:07.274+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.344 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:34:07.339+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.395 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:34:07.391+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.399 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:34:07.394+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.444 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:34:07.439+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.446 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:34:07.442+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.446 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:34:07.441+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.470 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:34:07.465+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.470 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:34:07.465+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.613 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:34:07.607+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.629 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:34:07.624+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:07.726 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:34:07.720+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:34:09.189 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:09.253 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 10 monclient: get_monmap_and_config 2026-02-24T18:34:09.253 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 10 monclient: build_initial_monmap 2026-02-24T18:34:09.253 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 10 monclient: monmap: 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:09.252234+0000 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:09.252234+0000 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:09.254 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:09.255 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:09.255 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.255 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 1 Processor -- start 2026-02-24T18:34:09.255 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 1 -- start start 2026-02-24T18:34:09.255 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.251+0000 7f4e784a9640 10 monclient: init 2026-02-24T18:34:09.263 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient: _add_conns ranks=[4,5,3,2,0,1] 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4e70152a40 0x7f4e700c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-b con 0x7f4e70152a40 addr v2:10.20.193.32:3300/0 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4e70152e20 0x7f4e700c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-c con 0x7f4e70152e20 addr v2:10.20.193.110:3300/0 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 0x7f4e7006e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-a con 0x7f4e7006dd40 addr v2:10.20.193.18:3300/0 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7006d5d0 con 0x7f4e7006dd40 2026-02-24T18:34:09.264 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7006e6d0 con 0x7f4e70152a40 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4e700c6d70 con 0x7f4e70152e20 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e784a9640 10 monclient(hunting): authenticate will time out at 1392.933434s 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.252+0000 7f4e75a1d640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4e70152e20 0x7f4e700c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 0x7f4e7006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4e70152a40 0x7f4e700c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 0x7f4e7006e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:57464/0 (socket says 10.20.193.18:57464) 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 1 -- 10.20.193.18:0/3852139923 learned_addr learned my addr 10.20.193.18:0/3852139923 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:09.265 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): get_auth_request con 0x7f4e7006dd40 auth_method 0 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): get_auth_request con 0x7f4e70152e20 auth_method 0 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): get_auth_request con 0x7f4e70152a40 auth_method 0 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.266 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.253+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_done global_id 4268 payload 274 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 -- 10.20.193.18:0/3852139923 >> v2:10.20.193.32:3300/0 conn(0x7f4e70152a40 msgr2=0x7f4e700c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 --2- 10.20.193.18:0/3852139923 >> v2:10.20.193.32:3300/0 conn(0x7f4e70152a40 0x7f4e700c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.267 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 -- 10.20.193.18:0/3852139923 >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 msgr2=0x7f4e7006e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 --2- 10.20.193.18:0/3852139923 >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 0x7f4e7006e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 10 monclient: _finish_hunting 0 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 monclient: found mon.noname-c 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.254+0000 7f4e75a1d640 1 -- 10.20.193.18:0/3852139923 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e700c7050 con 0x7f4e70152e20 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.255+0000 7f4e75a1d640 1 --2- 10.20.193.18:0/3852139923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e700c7e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e70071280 tx=0x7f4e6402ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50f7c4ce3186e3c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 1 -- 10.20.193.18:0/3852139923 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6403c070 con 0x7f4e70152e20 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:09.268 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: dump: 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:09.269 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: _finish_auth 0 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: _check_auth_tickets 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e784a9640 5 monclient: authenticate success, global_id 4268 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e784a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 1 -- 10.20.193.18:0/3852139923 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6402fa60 con 0x7f4e70152e20 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 1 -- 10.20.193.18:0/3852139923 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402fe30 con 0x7f4e70152e20 2026-02-24T18:34:09.270 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e7521c640 10 monclient: dump: 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:09.271 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e784a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.257+0000 7f4e784a9640 4 set_mon_vals no callback set 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 10 monclient: shutdown 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 -- 10.20.193.18:0/3852139923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 msgr2=0x7f4e700c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 --2- 10.20.193.18:0/3852139923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e700c7e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e70071280 tx=0x7f4e6402ede0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 -- 10.20.193.18:0/3852139923 shutdown_connections 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 --2- 10.20.193.18:0/3852139923 >> v2:10.20.193.18:3300/0 conn(0x7f4e7006dd40 0x7f4e7006e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.272 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 --2- 10.20.193.18:0/3852139923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e700c7e40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 --2- 10.20.193.18:0/3852139923 >> v2:10.20.193.32:3300/0 conn(0x7f4e70152a40 0x7f4e700c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 -- 10.20.193.18:0/3852139923 >> 10.20.193.18:0/3852139923 conn(0x7f4e700c9f40 msgr2=0x7f4e700ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 -- 10.20.193.18:0/3852139923 shutdown_connections 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 1 -- 10.20.193.18:0/3852139923 wait complete. 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 10 monclient: build_initial_monmap 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.258+0000 7f4e784a9640 10 monclient: monmap: 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:09.273 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 librados: starting msgr at 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 librados: starting objecter 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 Processor -- start 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 -- start start 2026-02-24T18:34:09.274 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 librados: setting wanted keys 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 librados: calling monclient init 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient: init 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4e7006dd40 0x7f4e7014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7006dd40 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 0x7f4e7015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-a con 0x7f4e70152a40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e701f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): picked mon.noname-c con 0x7f4e70152e20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.275 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7006d820 con 0x7f4e70152a40 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700c7610 con 0x7f4e7006dd40 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700c6960 con 0x7f4e70152e20 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e784a9640 10 monclient(hunting): authenticate will time out at 1392.940579s 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.259+0000 7f4e75a1d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 0x7f4e7015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e701f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4e7006dd40 0x7f4e7014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.276 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 0x7f4e7015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:57468/0 (socket says 10.20.193.18:57468) 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 1 -- 10.20.193.18:0/2464355260 learned_addr learned my addr 10.20.193.18:0/2464355260 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): get_auth_request con 0x7f4e70152a40 auth_method 0 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): get_auth_request con 0x7f4e70152e20 auth_method 0 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): get_auth_request con 0x7f4e7006dd40 auth_method 0 2026-02-24T18:34:09.277 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e75a1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.260+0000 7f4e7621e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:09.278 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 10 monclient(hunting): handle_auth_done global_id 4271 payload 824 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 1 -- 10.20.193.18:0/2464355260 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4e7006dd40 msgr2=0x7f4e7014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4e7006dd40 0x7f4e7014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 1 -- 10.20.193.18:0/2464355260 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 msgr2=0x7f4e7015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 0x7f4e7015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 10 monclient: _finish_hunting 0 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 1 monclient: found mon.noname-c 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.261+0000 7f4e76a1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.262+0000 7f4e76a1f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.279 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.262+0000 7f4e76a1f640 1 -- 10.20.193.18:0/2464355260 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701f4790 con 0x7f4e70152e20 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.263+0000 7f4e76a1f640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e701f3c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ebd0 tx=0x7f4e6c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.263+0000 7f4e56ffd640 10 client.?.objecter ms_handle_connect 0x7f4e70152e20 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.263+0000 7f4e56ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00cb30 con 0x7f4e70152e20 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: dump: 2026-02-24T18:34:09.282 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: _finish_auth 0 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:34:09.283 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00ccd0 con 0x7f4e70152e20 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c003cb0 con 0x7f4e70152e20 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e56ffd640 10 monclient: dump: 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:09.284 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e74a1b640 4 set_mon_vals no callback set 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 5 monclient: authenticate success, global_id 4271 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 10 monclient: _renew_subs 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701f4a70 con 0x7f4e70152e20 2026-02-24T18:34:09.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 10 client.4271.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 10 monclient: _renew_subs 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e70072cb0 con 0x7f4e70152e20 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.264+0000 7f4e784a9640 1 librados: init done 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.265+0000 7f4e784a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.265+0000 7f4e784a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.265+0000 7f4e784a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.265+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7015cc40 con 0x7f4e70152e20 2026-02-24T18:34:09.370 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.369+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f4e6c004030 con 0x7f4e70152e20 2026-02-24T18:34:09.371 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.369+0000 7f4e56ffd640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 0x7f4e4007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:09.371 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.370+0000 7f4e7621e640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 0x7f4e4007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:09.371 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.370+0000 7f4e7621e640 10 monclient: get_auth_request con 0x7f4e40078dc0 auth_method 0 2026-02-24T18:34:09.372 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.370+0000 7f4e7621e640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 0x7f4e4007b280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e701f17d0 tx=0x7f4e60009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:09.372 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.370+0000 7f4e56ffd640 10 client.4271.objecter ms_handle_connect 0x7f4e40078dc0 2026-02-24T18:34:09.378 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.376+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f4e6c05ecb0 con 0x7f4e70152e20 2026-02-24T18:34:09.378 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.376+0000 7f4e56ffd640 10 client.4271.objecter ms_dispatch2 0x7f4e70000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:09.378 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.376+0000 7f4e56ffd640 3 client.4271.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:09.379 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.376+0000 7f4e56ffd640 3 client.4271.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:09.379 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.377+0000 7f4e56ffd640 20 client.4271.objecter dump_active .. 0 homeless 2026-02-24T18:34:09.379 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.377+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4e6c010040 con 0x7f4e70152e20 2026-02-24T18:34:09.379 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.377+0000 7f4e56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:09.379 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.377+0000 7f4e56ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:09.468 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.466+0000 7f4e784a9640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:09.468 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.466+0000 7f4e784a9640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:09.468 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.466+0000 7f4e784a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:09.468 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.466+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e70156350 con 0x7f4e70152e20 2026-02-24T18:34:09.478 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.476+0000 7f4e56ffd640 1 -- 10.20.193.18:0/2464355260 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f4e6c062c70 con 0x7f4e70152e20 2026-02-24T18:34:09.478 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.476+0000 7f4e56ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:09.478 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.476+0000 7f4e56ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:09.478 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:09.478 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.478+0000 7f4e784a9640 10 librados: watch_flush enter 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.478+0000 7f4e784a9640 10 librados: watch_flush exit 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.478+0000 7f4e784a9640 20 client.4271.objecter shutdown clearing up homeless session... 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 10 client.4271.objecter successfully canceled tick 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 msgr2=0x7f4e4007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 0x7f4e4007b280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e701f17d0 tx=0x7f4e60009290 comp rx=0 tx=0).stop 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 10 monclient: shutdown 2026-02-24T18:34:09.481 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 msgr2=0x7f4e701f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e701f3c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ebd0 tx=0x7f4e6c00c490 comp rx=0 tx=0).stop 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 shutdown_connections 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4e40078dc0 0x7f4e4007b280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4e70152e20 0x7f4e701f3c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4e70152a40 0x7f4e7015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 --2- 10.20.193.18:0/2464355260 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4e7006dd40 0x7f4e7014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 >> 10.20.193.18:0/2464355260 conn(0x7f4e700c9f40 msgr2=0x7f4e700c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 shutdown_connections 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 -- 10.20.193.18:0/2464355260 wait complete. 2026-02-24T18:34:09.482 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:09.479+0000 7f4e784a9640 1 librados: shutdown 2026-02-24T18:34:09.489 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:11.736 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:20.335 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:11.907 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:20.502 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:11.946 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:20.558 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.002 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:20.613 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.058 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:20.658 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.116 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:20.710 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.174 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:20.776 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.223 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:20.805 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.228 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:20.823 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.258 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:20.875 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.265 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:20.850 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.266 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:20.852 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.274 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:20.887 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.328 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:20.942 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.338 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:20.950 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.342 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:20.942 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.356 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:20.968 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.361 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:20.978 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.439 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:21.034 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.462 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:21.048 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.497 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:21.100 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.539 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:21.121 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.541 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:21.137 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.551 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:21.161 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.581 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:21.197 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.583 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:21.196 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.592 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:21.176 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.592 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:21.178 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.610 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:21.209 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.647 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:21.258 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.652 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:21.266 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.677 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:21.281 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.684 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:21.296 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.684 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:21.301 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.724 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:21.319 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.781 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:21.367 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.782 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:21.384 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.819 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:21.414 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.850 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:21.462 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.851 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:21.433 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.866 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:21.483 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.871 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:21.481 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.878 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:21.477 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.909 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:21.493 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.912 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:21.499 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.925 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:21.536 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.937 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:21.551 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.984 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:21.601 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.988 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:21.582 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.995 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:21.599 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:12.996 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:21.608 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.046 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:21.648 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.087 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:21.673 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.098 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:21.693 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.114 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:21.726 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.129 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:21.711 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.130 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:21.746 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.141 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:21.741 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.176 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:21.785 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.190 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:21.774 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.190 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:21.776 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.201 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:21.815 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.204 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:21.816 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.248 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:21.864 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.252 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:21.846 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.273 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:21.885 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.279 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:21.883 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.309 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:21.911 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.361 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:21.947 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.372 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:21.967 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.378 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:21.990 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.401 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:22.017 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.405 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:22.004 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.417 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:22.000 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.449 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:22.059 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.465 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:22.078 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.467 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:22.054 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.471 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:22.056 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.481 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:22.093 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.512 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:22.128 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.516 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:22.110 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.542 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:22.147 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.547 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:22.159 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.573 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:22.175 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.640 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:22.226 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.642 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:22.254 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.646 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:22.241 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.664 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:22.280 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.677 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:22.276 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.696 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:22.278 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.727 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:22.337 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.729 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:22.343 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.748 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:22.335 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.752 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:22.337 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.760 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:22.372 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.775 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:22.391 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.780 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:22.374 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.809 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:22.414 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.821 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:22.433 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.837 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:22.439 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.910 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:22.522 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.914 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:22.500 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.924 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:22.519 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.932 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:22.548 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.950 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:22.549 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.971 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:22.553 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:13.992 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:22.606 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.002 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:22.611 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.026 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:22.613 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.030 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:22.615 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.036 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:22.648 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.037 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:22.654 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.047 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:22.642 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.074 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:22.678 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.095 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:22.707 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.112 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:22.715 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.174 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:22.786 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.188 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:22.774 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.195 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:22.811 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.202 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:22.797 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.245 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:22.827 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.257 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:22.871 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.280 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:22.890 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.309 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:22.925 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.313 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:22.925 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.315 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:22.915 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.316 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:22.903 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.320 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:22.905 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.325 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:22.919 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.343 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:22.948 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.369 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:22.981 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.376 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:22.978 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.459 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.075 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.461 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:23.048 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.520 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.102 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.524 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:23.138 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.534 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:23.147 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.557 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.167 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.572 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.189 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.577 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:23.190 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.578 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:23.173 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.595 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:23.182 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.599 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.184 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.600 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:23.213 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.613 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:23.213 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.615 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:23.220 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.632 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:23.231 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.647 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.259 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.693 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:23.287 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.697 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:23.309 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.736 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:23.322 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.736 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:23.348 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.743 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:23.345 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.780 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:23.382 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.803 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:23.405 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.812 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.429 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.831 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:23.433 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.831 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.441 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.835 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:23.437 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.854 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.470 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.867 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:23.480 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.870 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:23.465 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.872 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:23.485 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.882 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:23.496 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.885 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:23.489 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.894 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.477 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.903 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:23.498 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.917 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:23.516 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.922 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:23.522 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.936 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.552 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.941 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:23.536 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.947 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:23.542 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.955 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.538 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.976 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:23.562 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.988 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.572 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:14.993 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.575 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.002 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.618 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.002 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:23.614 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.003 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:23.597 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.031 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.615 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.032 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.644 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.040 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:23.652 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.044 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:23.639 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.045 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:23.657 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.054 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.638 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.072 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.685 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.080 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.664 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.084 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:23.669 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.096 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.708 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.118 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:23.704 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.120 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.737 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.146 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.763 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.151 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:23.768 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.163 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:23.749 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.191 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:23.805 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.208 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.818 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.211 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:23.825 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.260 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.869 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.260 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.843 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.266 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:23.870 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.267 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:23.849 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.277 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.893 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.278 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:23.864 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.282 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.892 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.301 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.917 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.306 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:23.922 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.320 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:23.915 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.322 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.932 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.325 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:23.929 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.326 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:23.921 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.327 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:23.937 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.357 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:23.962 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.379 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.992 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.385 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:23.997 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.399 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:24.004 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.404 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:24.008 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.437 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:24.023 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.487 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:24.100 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.492 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:24.105 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.560 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:24.146 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.724 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:24.310 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.732 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:24.318 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.848 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:24.434 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:15.853 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:24.440 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:17.103 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:17.164 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.162+0000 7fbc94e85640 10 monclient: get_monmap_and_config 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.162+0000 7fbc94e85640 10 monclient: build_initial_monmap 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.162+0000 7fbc94e85640 10 monclient: monmap: 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:17.163752+0000 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:17.163752+0000 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.165 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.166 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 Processor -- start 2026-02-24T18:34:17.167 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 -- start start 2026-02-24T18:34:17.167 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient: init 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient: _add_conns ranks=[5,4,3,1,2,0] 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbc901539a0 0x7fbc900c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-c con 0x7fbc901539a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fbc90153d80 0x7fbc90074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-b con 0x7fbc90153d80 addr v2:10.20.193.32:3300/0 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fbc900c8b00 0x7fbc900c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-a con 0x7fbc900c8b00 addr v2:10.20.193.18:3300/0 2026-02-24T18:34:17.175 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbc900c5cd0 con 0x7fbc900c8b00 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbc900c9490 con 0x7fbc90153d80 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbc900c5710 con 0x7fbc901539a0 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.163+0000 7fbc94e85640 10 monclient(hunting): authenticate will time out at 1400.845006s 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8ed76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fbc900c8b00 0x7fbc900c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbc901539a0 0x7fbc900c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8dd74640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fbc90153d80 0x7fbc90074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbc901539a0 0x7fbc900c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:47892/0 (socket says 10.20.193.18:47892) 2026-02-24T18:34:17.176 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 1 -- 10.20.193.18:0/2931118273 learned_addr learned my addr 10.20.193.18:0/2931118273 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 10 monclient(hunting): get_auth_request con 0x7fbc901539a0 auth_method 0 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8ed76640 10 monclient(hunting): get_auth_request con 0x7fbc900c8b00 auth_method 0 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8dd74640 10 monclient(hunting): get_auth_request con 0x7fbc90153d80 auth_method 0 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.164+0000 7fbc8dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.177 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 -- 10.20.193.18:0/2931118273 >> v2:10.20.193.32:3300/0 conn(0x7fbc90153d80 msgr2=0x7fbc90074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 --2- 10.20.193.18:0/2931118273 >> v2:10.20.193.32:3300/0 conn(0x7fbc90153d80 0x7fbc90074960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 -- 10.20.193.18:0/2931118273 >> v2:10.20.193.18:3300/0 conn(0x7fbc900c8b00 msgr2=0x7fbc900c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:17.178 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 --2- 10.20.193.18:0/2931118273 >> v2:10.20.193.18:3300/0 conn(0x7fbc900c8b00 0x7fbc900c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient: _finish_hunting 0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 monclient: found mon.noname-c 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.165+0000 7fbc8e575640 1 -- 10.20.193.18:0/2931118273 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc900c59f0 con 0x7fbc901539a0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.166+0000 7fbc8e575640 1 --2- 10.20.193.18:0/2931118273 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc901539a0 0x7fbc900c7270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc9006f660 tx=0x7fbc8402edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ffe1e1468daea8c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 1 -- 10.20.193.18:0/2931118273 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8403c070 con 0x7fbc901539a0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: dump: 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:17.179 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: _finish_auth 0 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: _check_auth_tickets 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:17.180 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 1 -- 10.20.193.18:0/2931118273 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc8402fab0 con 0x7fbc901539a0 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 1 -- 10.20.193.18:0/2931118273 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84037440 con 0x7fbc901539a0 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 5 monclient: authenticate success, global_id 4274 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 4 set_mon_vals no callback set 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc8d573640 10 monclient: dump: 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.181 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 10 monclient: shutdown 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 -- 10.20.193.18:0/2931118273 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc901539a0 msgr2=0x7fbc900c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 --2- 10.20.193.18:0/2931118273 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc901539a0 0x7fbc900c7270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc9006f660 tx=0x7fbc8402edf0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 -- 10.20.193.18:0/2931118273 shutdown_connections 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 --2- 10.20.193.18:0/2931118273 >> v2:10.20.193.18:3300/0 conn(0x7fbc900c8b00 0x7fbc900c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 --2- 10.20.193.18:0/2931118273 >> v2:10.20.193.32:3300/0 conn(0x7fbc90153d80 0x7fbc90074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 --2- 10.20.193.18:0/2931118273 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc901539a0 0x7fbc900c7270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 -- 10.20.193.18:0/2931118273 >> 10.20.193.18:0/2931118273 conn(0x7fbc900c9f40 msgr2=0x7fbc900ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 -- 10.20.193.18:0/2931118273 shutdown_connections 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.169+0000 7fbc94e85640 1 -- 10.20.193.18:0/2931118273 wait complete. 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient: build_initial_monmap 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient: monmap: 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:17.183 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 librados: starting msgr at 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 librados: starting objecter 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 Processor -- start 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 -- start start 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 librados: setting wanted keys 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 librados: calling monclient init 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient: init 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:17.184 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc900c8b00 0x7fbc9006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-c con 0x7fbc900c8b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbc901539a0 0x7fbc90073da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-a con 0x7fbc901539a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): picked mon.noname-b con 0x7fbc90153d80 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900c85e0 con 0x7fbc901539a0 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900c8300 con 0x7fbc90153d80 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900c77b0 con 0x7fbc900c8b00 2026-02-24T18:34:17.185 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.170+0000 7fbc94e85640 10 monclient(hunting): authenticate will time out at 1400.851973s 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbc901539a0 0x7fbc90073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8e575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc900c8b00 0x7fbc9006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:55504/0 (socket says 10.20.193.18:55504) 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 1 -- 10.20.193.18:0/3806795114 learned_addr learned my addr 10.20.193.18:0/3806795114 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): get_auth_request con 0x7fbc90153d80 auth_method 0 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): get_auth_request con 0x7fbc901539a0 auth_method 0 2026-02-24T18:34:17.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8e575640 10 monclient(hunting): get_auth_request con 0x7fbc900c8b00 auth_method 0 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.171+0000 7fbc8dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.172+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.172+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.172+0000 7fbc8e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 10 monclient(hunting): handle_auth_done global_id 4276 payload 824 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 -- 10.20.193.18:0/3806795114 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc900c8b00 msgr2=0x7fbc9006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc900c8b00 0x7fbc9006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 -- 10.20.193.18:0/3806795114 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbc901539a0 msgr2=0x7fbc90073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbc901539a0 0x7fbc90073da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 10 monclient: _finish_hunting 0 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 monclient: found mon.noname-b 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:34:17.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.173+0000 7fbc8ed76640 1 -- 10.20.193.18:0/3806795114 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9006d950 con 0x7fbc90153d80 2026-02-24T18:34:17.190 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.174+0000 7fbc8ed76640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000eed0 tx=0x7fbc8000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.174+0000 7fbc76ffd640 10 client.?.objecter ms_handle_connect 0x7fbc90153d80 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.174+0000 7fbc76ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80018020 con 0x7fbc90153d80 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 10 monclient: dump: 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:17.191 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 10 monclient: _finish_auth 0 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 5 monclient: authenticate success, global_id 4276 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _renew_subs 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9006dc30 con 0x7fbc90153d80 2026-02-24T18:34:17.192 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 client.4276.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _renew_subs 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9006e1f0 con 0x7fbc90153d80 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 1 librados: init done 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.175+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc900c52d0 con 0x7fbc90153d80 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc80026430 con 0x7fbc90153d80 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc800267e0 con 0x7fbc90153d80 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:17.193 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc76ffd640 10 monclient: dump: 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:17.194 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.176+0000 7fbc8cd72640 4 set_mon_vals no callback set 2026-02-24T18:34:17.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.283+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7fbc80026980 con 0x7fbc90153d80 2026-02-24T18:34:17.285 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.283+0000 7fbc76ffd640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 0x7fbc5c07b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:17.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.284+0000 7fbc8e575640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 0x7fbc5c07b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:17.286 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.284+0000 7fbc8e575640 10 monclient: get_auth_request con 0x7fbc5c078d70 auth_method 0 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.284+0000 7fbc8e575640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 0x7fbc5c07b230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc90070f60 tx=0x7fbc84039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.284+0000 7fbc76ffd640 10 client.4276.objecter ms_handle_connect 0x7fbc5c078d70 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.285+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7fbc8005ecd0 con 0x7fbc90153d80 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.285+0000 7fbc76ffd640 10 client.4276.objecter ms_dispatch2 0x7fbc90000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.285+0000 7fbc76ffd640 3 client.4276.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.285+0000 7fbc76ffd640 3 client.4276.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:17.287 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.285+0000 7fbc76ffd640 20 client.4276.objecter dump_active .. 0 homeless 2026-02-24T18:34:17.293 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.291+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbc80022030 con 0x7fbc90153d80 2026-02-24T18:34:17.293 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.291+0000 7fbc76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:17.293 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.291+0000 7fbc76ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:17.380 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.378+0000 7fbc94e85640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:17.380 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.378+0000 7fbc94e85640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:17.380 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.378+0000 7fbc94e85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:17.380 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.378+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc901497b0 con 0x7fbc90153d80 2026-02-24T18:34:17.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.390+0000 7fbc76ffd640 1 -- 10.20.193.18:0/3806795114 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7fbc80062c90 con 0x7fbc90153d80 2026-02-24T18:34:17.392 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.390+0000 7fbc76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:17.393 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.390+0000 7fbc76ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:17.393 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:17.393 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 10 librados: watch_flush enter 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 10 librados: watch_flush exit 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 20 client.4276.objecter shutdown clearing up homeless session... 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 10 client.4276.objecter successfully canceled tick 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 msgr2=0x7fbc5c07b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 0x7fbc5c07b230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc90070f60 tx=0x7fbc84039040 comp rx=0 tx=0).stop 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 10 monclient: shutdown 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:17.395 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 msgr2=0x7fbc900742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000eed0 tx=0x7fbc8000b480 comp rx=0 tx=0).stop 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 shutdown_connections 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbc5c078d70 0x7fbc5c07b230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbc90153d80 0x7fbc900742e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbc901539a0 0x7fbc90073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 --2- 10.20.193.18:0/3806795114 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbc900c8b00 0x7fbc9006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 >> 10.20.193.18:0/3806795114 conn(0x7fbc900c9f40 msgr2=0x7fbc9014f680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 shutdown_connections 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 -- 10.20.193.18:0/3806795114 wait complete. 2026-02-24T18:34:17.396 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:17.392+0000 7fbc94e85640 1 librados: shutdown 2026-02-24T18:34:17.402 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:25.015 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 10 monclient: get_monmap_and_config 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 10 monclient: build_initial_monmap 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 10 monclient: monmap: 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:25.078118+0000 2026-02-24T18:34:25.079 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:25.078118+0000 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 1 Processor -- start 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 1 -- start start 2026-02-24T18:34:25.080 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.077+0000 7f280021d640 10 monclient: init 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient: _add_conns ranks=[0,2,1,3,4,5] 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f27f8152a40 addr v1:10.20.193.18:6789/0 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f27f8152e20 addr v1:10.20.193.110:6789/0 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f27f806e450 addr v1:10.20.193.32:6789/0 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f80c6d70 con 0x7f27f8152a40 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f80c7050 con 0x7f27f806e450 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f80c7330 con 0x7f27f8152e20 2026-02-24T18:34:25.083 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f280021d640 10 monclient(hunting): authenticate will time out at 1408.759260s 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f27fd791640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f27f8152e20 0x7f27f806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:56900/0 (socket says 10.20.193.18:56900) 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.078+0000 7f27fd791640 1 -- 10.20.193.18:0/23746423 learned_addr learned my addr 10.20.193.18:0/23746423 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533897724 0 0) 0x7f27f80c7330 con 0x7f27f8152e20 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _negotiate 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): my global_id is 4280 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27e0003620 con 0x7f27f8152e20 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049138312 0 0) 0x7f27f80c7050 con 0x7f27f806e450 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _negotiate 2026-02-24T18:34:25.084 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): my global_id is 4279 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f80c7330 con 0x7f27f806e450 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703833658 0 0) 0x7f27f80c6d70 con 0x7f27f8152a40 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.080+0000 7f27fcf90640 10 monclient(hunting): _negotiate 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.081+0000 7f27fcf90640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.081+0000 7f27fcf90640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.081+0000 7f27fcf90640 10 monclient(hunting): my global_id is 4275 2026-02-24T18:34:25.085 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.081+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f80c7050 con 0x7f27f8152a40 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 848428155 0 0) 0x7f27e0003620 con 0x7f27f8152e20 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 >> v1:10.20.193.32:6789/0 conn(0x7f27f806e450 legacy=0x7f27f815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 >> v1:10.20.193.18:6789/0 conn(0x7f27f8152a40 legacy=0x7f27f80c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _finish_hunting 0 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f80c7610 con 0x7f27f8152e20 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _finish_auth 0 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _check_auth_tickets 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f27ec002f00 con 0x7f27f8152e20 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:25.092 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: dump: 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 >> v1:10.20.193.110:6789/0 conn(0x7f27f8152e20 legacy=0x7f27f806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27e0003fb0 0x7f27e0004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): picked mon.b con 0x7f27e0003fb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e00044c0 0x7f27e0004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): picked mon.a con 0x7f27e00044c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 0x7f27e000e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.093 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): picked mon.c con 0x7f27e000bd40 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27e000e7a0 con 0x7f27e00044c0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27e000e950 con 0x7f27e0003fb0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27e000eb50 con 0x7f27e000bd40 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.082+0000 7f27fcf90640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fe793640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 0x7f27e000e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fd791640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e00044c0 0x7f27e0004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fe793640 10 monclient(hunting): get_auth_request con 0x7f27e000bd40 auth_method 0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fe793640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fe793640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fe793640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.083+0000 7f27fdf92640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27e0003fb0 0x7f27e0004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): get_auth_request con 0x7f27e00044c0 auth_method 0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): get_auth_request con 0x7f27e0003fb0 auth_method 0 2026-02-24T18:34:25.094 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.084+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 10 monclient(hunting): handle_auth_done global_id 4280 payload 293 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 -- 10.20.193.18:0/23746423 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27e0003fb0 msgr2=0x7f27e0004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27e0003fb0 0x7f27e0004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 -- 10.20.193.18:0/23746423 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e00044c0 msgr2=0x7f27e0004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e00044c0 0x7f27e0004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 10 monclient: _finish_hunting 0 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 monclient: found mon.c 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 -- 10.20.193.18:0/23746423 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e0003620 con 0x7f27e000bd40 2026-02-24T18:34:25.095 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 10 monclient: _finish_auth 0 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 10 monclient: _check_auth_tickets 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f280021d640 5 monclient: authenticate success, global_id 4280 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f280021d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.085+0000 7f27fe793640 1 -- 10.20.193.18:0/23746423 reap_dead start 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.086+0000 7f27fe793640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 0x7f27e000e130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c320 tx=0x7f27f402f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7877b23412f3a26 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f40038b0 con 0x7f27e000bd40 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: dump: 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f4003c70 con 0x7f27e000bd40 2026-02-24T18:34:25.096 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f280021d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f280021d640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f280021d640 4 set_mon_vals no callback set 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 1 -- 10.20.193.18:0/23746423 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4005310 con 0x7f27e000bd40 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.087+0000 7f27fcf90640 10 monclient: dump: 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 10 monclient: shutdown 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 msgr2=0x7f27e000e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 0x7f27e000e130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c320 tx=0x7f27f402f630 comp rx=0 tx=0).stop 2026-02-24T18:34:25.097 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- 10.20.193.18:0/23746423 shutdown_connections 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 --2- 10.20.193.18:0/23746423 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27e000bd40 0x7f27e000e130 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- 10.20.193.18:0/23746423 >> 10.20.193.18:0/23746423 conn(0x7f27f80c9f40 msgr2=0x7f27f80ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- 10.20.193.18:0/23746423 shutdown_connections 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- 10.20.193.18:0/23746423 wait complete. 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 10 monclient: build_initial_monmap 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 10 monclient: monmap: 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 librados: starting msgr at 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 librados: starting objecter 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 Processor -- start 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 -- start start 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 librados: setting wanted keys 2026-02-24T18:34:25.098 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 1 librados: calling monclient init 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.088+0000 7f280021d640 10 monclient: init 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 0x7f27f81f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): picked mon.noname-a con 0x7f27e000bd40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 0x7f27f815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): picked mon.noname-c con 0x7f27f80cb950 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27f80cbdd0 0x7f27f815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): picked mon.noname-b con 0x7f27f80cbdd0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27f80c6960 con 0x7f27e000bd40 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27f806d5d0 con 0x7f27f80cbdd0 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27f80c6ae0 con 0x7f27f80cb950 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f280021d640 10 monclient(hunting): authenticate will time out at 1408.770347s 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 0x7f27f81f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fd791640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 0x7f27f815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fe793640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27f80cbdd0 0x7f27f815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 0x7f27f81f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55582/0 (socket says 10.20.193.18:55582) 2026-02-24T18:34:25.099 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fd791640 10 monclient(hunting): get_auth_request con 0x7f27f80cb950 auth_method 0 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 1 -- 10.20.193.18:0/50294147 learned_addr learned my addr 10.20.193.18:0/50294147 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fd791640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fd791640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fd791640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 10 monclient(hunting): get_auth_request con 0x7f27e000bd40 auth_method 0 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fdf92640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fe793640 10 monclient(hunting): get_auth_request con 0x7f27f80cbdd0 auth_method 0 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fe793640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fe793640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.089+0000 7f27fe793640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fd791640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fdf92640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:25.100 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.090+0000 7f27fe793640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 10 monclient(hunting): handle_auth_done global_id 4283 payload 824 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 -- 10.20.193.18:0/50294147 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27f80cbdd0 msgr2=0x7f27f815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27f80cbdd0 0x7f27f815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 -- 10.20.193.18:0/50294147 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 msgr2=0x7f27f81f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 0x7f27f81f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 10 monclient: _finish_hunting 0 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 monclient: found mon.noname-c 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.091+0000 7f27fd791640 1 -- 10.20.193.18:0/50294147 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f814db60 con 0x7f27f80cb950 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.092+0000 7f27fd791640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 0x7f27f815b210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27ec003070 tx=0x7f27ec007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.092+0000 7f27e67fc640 10 client.?.objecter ms_handle_connect 0x7f27f80cb950 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.092+0000 7f27e67fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00f750 con 0x7f27f80cb950 2026-02-24T18:34:25.102 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 10 monclient: dump: 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 10 monclient: _finish_auth 0 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f27e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 5 monclient: authenticate success, global_id 4283 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 10 monclient: _renew_subs 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f815a170 con 0x7f27f80cb950 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 10 client.4283.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:25.103 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 10 monclient: _renew_subs 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f815a730 con 0x7f27f80cb950 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.093+0000 7f280021d640 1 librados: init done 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ec018070 con 0x7f27f80cb950 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f280021d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00faf0 con 0x7f27f80cb950 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f280021d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f280021d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e7fff640 4 set_mon_vals no callback set 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f815cc40 con 0x7f27f80cb950 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.094+0000 7f27e67fc640 10 monclient: dump: 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:25.104 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:25.105 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:25.105 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:25.205 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.204+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f27ec00fc90 con 0x7f27f80cb950 2026-02-24T18:34:25.206 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.204+0000 7f27e67fc640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 0x7f27c807b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:25.206 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.205+0000 7f27fdf92640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 0x7f27c807b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:25.206 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.205+0000 7f27fdf92640 10 monclient: get_auth_request con 0x7f27c8078d70 auth_method 0 2026-02-24T18:34:25.207 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.205+0000 7f27e67fc640 10 client.4283.objecter ms_handle_connect 0x7f27c8078d70 2026-02-24T18:34:25.207 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.205+0000 7f27fdf92640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 0x7f27c807b230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f27e8002410 tx=0x7f27e8005100 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f27ec05e1e0 con 0x7f27f80cb950 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 10 client.4283.objecter ms_dispatch2 0x7f27f8000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 3 client.4283.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 3 client.4283.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 20 client.4283.objecter dump_active .. 0 homeless 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f27ec0621a0 con 0x7f27f80cb950 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:25.214 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.212+0000 7f27e67fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:25.303 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.302+0000 7f280021d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:25.303 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.302+0000 7f280021d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:25.303 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.302+0000 7f280021d640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:34:25.303 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.302+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27f8156350 con 0x7f27f80cb950 2026-02-24T18:34:25.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.314+0000 7f27e67fc640 1 -- 10.20.193.18:0/50294147 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f27ec0664a0 con 0x7f27f80cb950 2026-02-24T18:34:25.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.314+0000 7f27e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:25.315 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.314+0000 7f27e67fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:25.316 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:25.316 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:25.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 10 librados: watch_flush enter 2026-02-24T18:34:25.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 10 librados: watch_flush exit 2026-02-24T18:34:25.318 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 20 client.4283.objecter shutdown clearing up homeless session... 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 10 client.4283.objecter successfully canceled tick 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 msgr2=0x7f27c807b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 0x7f27c807b230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f27e8002410 tx=0x7f27e8005100 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 10 monclient: shutdown 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 msgr2=0x7f27f815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 0x7f27f815b210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27ec003070 tx=0x7f27ec007500 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 shutdown_connections 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f27c8078d70 0x7f27c807b230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27f80cbdd0 0x7f27f815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27f80cb950 0x7f27f815b210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 --2- 10.20.193.18:0/50294147 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27e000bd40 0x7f27f81f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.316+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 >> 10.20.193.18:0/50294147 conn(0x7f27f80c9f40 msgr2=0x7f27f806e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.317+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 shutdown_connections 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.317+0000 7f280021d640 1 -- 10.20.193.18:0/50294147 wait complete. 2026-02-24T18:34:25.319 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:25.317+0000 7f280021d640 1 librados: shutdown 2026-02-24T18:34:25.327 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:26.306 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:34.908 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.407 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:35.007 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.439 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:35.034 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.591 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:35.204 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.601 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:35.185 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.771 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:35.382 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.804 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:35.398 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.860 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:35.442 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:26.865 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:35.482 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.031 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:35.640 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.065 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:35.681 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.120 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:35.732 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.153 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:35.758 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.224 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:35.838 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.315 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:35.901 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:27.434 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:36.020 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.598 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:37.200 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.626 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:37.229 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.629 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:37.231 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.699 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:37.299 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.723 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:37.322 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.726 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:37.325 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.746 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:37.341 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.772 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:37.367 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.775 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:37.370 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.915 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:37.499 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.923 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:37.536 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.946 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:37.530 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.947 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:37.560 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.949 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:37.533 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:28.950 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:37.563 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.033 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:37.645 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.073 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:37.684 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.075 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:37.687 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.094 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:37.689 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.160 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:37.777 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.185 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:37.801 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.187 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:37.804 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.199 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:37.781 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.375 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:37.969 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.378 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:37.972 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.382 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:37.999 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.421 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:38.031 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.448 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:38.057 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.450 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:38.060 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.472 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:38.055 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.475 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:38.058 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.494 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:38.106 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.520 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:38.124 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.538 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:38.150 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.541 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:38.153 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.554 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:38.159 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.557 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:38.161 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.575 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:38.189 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.596 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:38.209 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.598 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:38.212 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.608 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:38.194 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.639 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:38.225 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.642 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:38.228 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.662 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:38.278 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.665 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:38.281 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:29.792 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:38.378 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.065 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:38.651 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.068 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:38.654 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.327 INFO:tasks.ceph.osd.2.trial110.stderr:==00:00:00:38.929 45414== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.484 INFO:tasks.ceph.osd.10.trial110.stderr:==00:00:00:39.079 45416== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.554 INFO:tasks.ceph.osd.6.trial110.stderr:==00:00:00:39.154 45415== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.719 INFO:tasks.ceph.osd.1.trial032.stderr:==00:00:00:39.331 44816== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.764 INFO:tasks.ceph.osd.11.trial112.stderr:==00:00:00:39.349 45530== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:30.840 INFO:tasks.ceph.osd.5.trial032.stderr:==00:00:00:39.451 44817== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.014 INFO:tasks.ceph.osd.4.trial018.stderr:==00:00:00:39.630 45705== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.153 INFO:tasks.ceph.osd.9.trial032.stderr:==00:00:00:39.763 44818== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.184 INFO:tasks.ceph.osd.14.trial110.stderr:==00:00:00:39.778 45417== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.231 INFO:tasks.ceph.osd.8.trial018.stderr:==00:00:00:39.845 45710== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.248 INFO:tasks.ceph.osd.13.trial032.stderr:==00:00:00:39.852 44819== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.274 INFO:tasks.ceph.osd.15.trial112.stderr:==00:00:00:39.857 45531== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.387 INFO:tasks.ceph.osd.12.trial018.stderr:==00:00:00:39.999 45713== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.447 INFO:tasks.ceph.osd.3.trial112.stderr:==00:00:00:40.033 45528== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.519 INFO:tasks.ceph.osd.0.trial018.stderr:==00:00:00:40.136 45706== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:31.659 INFO:tasks.ceph.osd.7.trial112.stderr:==00:00:00:40.246 45529== Warning: unimplemented fcntl command: 1038 2026-02-24T18:34:32.940 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:33.000 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 10 monclient: get_monmap_and_config 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 10 monclient: build_initial_monmap 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 10 monclient: monmap: 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:32.999950+0000 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:32.999950+0000 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:33.001 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:33.002 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:33.002 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:33.002 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:33.002 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:33.002 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.004 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 1 Processor -- start 2026-02-24T18:34:33.004 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 1 -- start start 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:32.999+0000 7f644e91b640 10 monclient: init 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient: _add_conns ranks=[2,4,1,5,0,3] 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6448152630 addr v1:10.20.193.110:6789/0 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f64480cd390 0x7f64480cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-b con 0x7f64480cd390 addr v2:10.20.193.32:3300/0 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6448152a10 addr v1:10.20.193.32:6789/0 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480cc010 con 0x7f6448152a10 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480cc2f0 con 0x7f6448152630 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f64480ccf00 con 0x7f64480cd390 2026-02-24T18:34:33.005 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f644e91b640 10 monclient(hunting): authenticate will time out at 1416.681210s 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f6447fff640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f6448152630 0x7f64480c8680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:56428/0 (socket says 10.20.193.18:56428) 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.000+0000 7f6447fff640 1 -- 10.20.193.18:0/3085650030 learned_addr learned my addr 10.20.193.18:0/3085650030 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.001+0000 7f64477fe640 1 --2- 10.20.193.18:0/3085650030 >> v2:10.20.193.32:3300/0 conn(0x7f64480cd390 0x7f64480cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f64477fe640 10 monclient(hunting): get_auth_request con 0x7f64480cd390 auth_method 0 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f64477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f64477fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f64477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 1 -- 10.20.193.18:0/3085650030 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336176971 0 0) 0x7f64480cc2f0 con 0x7f6448152630 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:33.006 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 10 monclient(hunting): my global_id is 4286 2026-02-24T18:34:33.007 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.002+0000 7f6446ffd640 1 -- 10.20.193.18:0/3085650030 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f642c003620 con 0x7f6448152630 2026-02-24T18:34:33.007 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.003+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:33.007 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.003+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:33.007 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.003+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 10 monclient(hunting): handle_auth_done global_id 4285 payload 274 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 1 -- 10.20.193.18:0/3085650030 >> v1:10.20.193.110:6789/0 conn(0x7f6448152630 legacy=0x7f64480c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 1 -- 10.20.193.18:0/3085650030 >> v1:10.20.193.32:6789/0 conn(0x7f6448152a10 legacy=0x7f644806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 10 monclient: _finish_hunting 0 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 1 monclient: found mon.noname-b 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:34:33.008 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.005+0000 7f64477fe640 1 -- 10.20.193.18:0/3085650030 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480cc5d0 con 0x7f64480cd390 2026-02-24T18:34:33.010 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.009+0000 7f64477fe640 1 --2- 10.20.193.18:0/3085650030 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f64480cb1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64380098a0 tx=0x7f643802edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70263e8dfd48963 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:33.017 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 1 -- 10.20.193.18:0/3085650030 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643803c070 con 0x7f64480cd390 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: dump: 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:33.018 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: _finish_auth 0 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 1 -- 10.20.193.18:0/3085650030 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643802fae0 con 0x7f64480cd390 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 1 -- 10.20.193.18:0/3085650030 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438037440 con 0x7f64480cd390 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f6446ffd640 10 monclient: dump: 2026-02-24T18:34:33.019 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 5 monclient: authenticate success, global_id 4285 2026-02-24T18:34:33.020 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 4 set_mon_vals no callback set 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 10 monclient: shutdown 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 1 -- 10.20.193.18:0/3085650030 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 msgr2=0x7f64480cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.011+0000 7f644e91b640 1 --2- 10.20.193.18:0/3085650030 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f64480cb1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64380098a0 tx=0x7f643802edd0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 -- 10.20.193.18:0/3085650030 shutdown_connections 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 --2- 10.20.193.18:0/3085650030 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f64480cb1c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 -- 10.20.193.18:0/3085650030 >> 10.20.193.18:0/3085650030 conn(0x7f64480ce330 msgr2=0x7f64480ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 -- 10.20.193.18:0/3085650030 shutdown_connections 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 -- 10.20.193.18:0/3085650030 wait complete. 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 10 monclient: build_initial_monmap 2026-02-24T18:34:33.021 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 10 monclient: monmap: 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 librados: starting msgr at 2026-02-24T18:34:33.022 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 librados: starting objecter 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 Processor -- start 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 -- start start 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 librados: setting wanted keys 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 1 librados: calling monclient init 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.012+0000 7f644e91b640 10 monclient: init 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f644815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-b con 0x7f64480cd390 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-a con 0x7f6448152630 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6448152a10 0x7f64481f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:33.023 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): picked mon.noname-c con 0x7f6448152a10 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f64480cbc00 con 0x7f6448152630 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64480ccb90 con 0x7f64480cd390 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f64480cbd80 con 0x7f6448152a10 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644e91b640 10 monclient(hunting): authenticate will time out at 1416.694236s 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f6447fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f644815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644ce91640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6448152a10 0x7f64481f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:33.024 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42652/0 (socket says 10.20.193.18:42652) 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 1 -- 10.20.193.18:0/3838853378 learned_addr learned my addr 10.20.193.18:0/3838853378 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 10 monclient(hunting): get_auth_request con 0x7f6448152630 auth_method 0 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f64477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f6447fff640 10 monclient(hunting): get_auth_request con 0x7f64480cd390 auth_method 0 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f6447fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f6447fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f6447fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644ce91640 10 monclient(hunting): get_auth_request con 0x7f6448152a10 auth_method 0 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644ce91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644ce91640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:33.025 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.013+0000 7f644ce91640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f64477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f6447fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f6447fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f6447fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f644ce91640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f644ce91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.014+0000 7f644ce91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 -- 10.20.193.18:0/3838853378 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6448152a10 msgr2=0x7f64481f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.026 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6448152a10 0x7f64481f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 -- 10.20.193.18:0/3838853378 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 msgr2=0x7f644815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f644815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 10 monclient: _finish_hunting 0 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 monclient: found mon.noname-a 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:34:33.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.015+0000 7f64477fe640 1 -- 10.20.193.18:0/3838853378 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6448159fc0 con 0x7f6448152630 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.017+0000 7f64477fe640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6438009130 tx=0x7f64380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.017+0000 7f6444ff9640 10 client.?.objecter ms_handle_connect 0x7f6448152630 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.017+0000 7f6444ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643803c040 con 0x7f6448152630 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:34:33.029 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: dump: 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: _finish_auth 0 2026-02-24T18:34:33.030 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438042630 con 0x7f6448152630 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 5 monclient: authenticate success, global_id 4281 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _renew_subs 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644815a2a0 con 0x7f6448152630 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438042a20 con 0x7f6448152630 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f6444ff9640 10 monclient: dump: 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:33.031 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f64467fc640 4 set_mon_vals no callback set 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _renew_subs 2026-02-24T18:34:33.032 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644815a860 con 0x7f6448152630 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 1 librados: init done 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:33.033 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.019+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64480cb1c0 con 0x7f6448152630 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.030+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f64380413b0 con 0x7f6448152630 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.031+0000 7f6444ff9640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 0x7f641807b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.031+0000 7f6447fff640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 0x7f641807b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.031+0000 7f6447fff640 10 monclient: get_auth_request con 0x7f6418078d70 auth_method 0 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.031+0000 7f6447fff640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 0x7f641807b230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6434005980 tx=0x7f6434007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:33.034 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.031+0000 7f6444ff9640 10 client.4281.objecter ms_handle_connect 0x7f6418078d70 2026-02-24T18:34:33.035 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.033+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f643802fa20 con 0x7f6448152630 2026-02-24T18:34:33.035 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.033+0000 7f6444ff9640 10 client.4281.objecter ms_dispatch2 0x7f6448000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:33.035 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.033+0000 7f6444ff9640 3 client.4281.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:33.035 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.033+0000 7f6444ff9640 3 client.4281.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:33.035 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.033+0000 7f6444ff9640 20 client.4281.objecter dump_active .. 0 homeless 2026-02-24T18:34:33.141 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.140+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6438086fe0 con 0x7f6448152630 2026-02-24T18:34:33.142 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.140+0000 7f6444ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:33.142 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.140+0000 7f6444ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:33.232 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.231+0000 7f644e91b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:33.232 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.231+0000 7f644e91b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:33.232 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.231+0000 7f644e91b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:33.232 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.231+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6448159830 con 0x7f6448152630 2026-02-24T18:34:33.242 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.241+0000 7f6444ff9640 1 -- 10.20.193.18:0/3838853378 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f643808b2e0 con 0x7f6448152630 2026-02-24T18:34:33.242 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.241+0000 7f6444ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:33.242 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.241+0000 7f6444ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:33.243 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:33.243 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 10 librados: watch_flush enter 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 10 librados: watch_flush exit 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 20 client.4281.objecter shutdown clearing up homeless session... 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 10 client.4281.objecter successfully canceled tick 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 msgr2=0x7f641807b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 0x7f641807b230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6434005980 tx=0x7f6434007dd0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 10 monclient: shutdown 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 msgr2=0x7f644814d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.243+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6438009130 tx=0x7f64380047c0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 shutdown_connections 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6418078d70 0x7f641807b230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.246 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6448152a10 0x7f64481f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6448152630 0x7f644814d140 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 --2- 10.20.193.18:0/3838853378 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f64480cd390 0x7f644815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 >> 10.20.193.18:0/3838853378 conn(0x7f64480ce330 msgr2=0x7f64480c5830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 shutdown_connections 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 -- 10.20.193.18:0/3838853378 wait complete. 2026-02-24T18:34:33.247 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:33.244+0000 7f644e91b640 1 librados: shutdown 2026-02-24T18:34:33.255 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:38.979 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:16.924 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:38.999 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:16.945 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.008 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:16.954 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.032 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:16.978 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.039 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:16.948 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.072 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:16.980 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.128 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:17.074 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.156 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:17.102 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:39.192 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:17.100 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:40.868 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 10 monclient: get_monmap_and_config 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 10 monclient: build_initial_monmap 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 10 monclient: monmap: 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:40.928952+0000 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:40.928952+0000 2026-02-24T18:34:40.930 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 1 Processor -- start 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 1 -- start start 2026-02-24T18:34:40.931 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.928+0000 7f5b077a9640 10 monclient: init 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient: _add_conns ranks=[0,1,3,4,2,5] 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b001539a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5b00153d80 addr v1:10.20.193.32:6789/0 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f5b000c8b00 0x7f5b000c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-a con 0x7f5b000c8b00 addr v2:10.20.193.18:3300/0 2026-02-24T18:34:40.934 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b00075070 con 0x7f5b001539a0 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b000c9490 con 0x7f5b00153d80 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5b000c5cd0 con 0x7f5b000c8b00 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b077a9640 10 monclient(hunting): authenticate will time out at 1424.610080s 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b04d1d640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f5b00153d80 0x7f5b00074960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:57080/0 (socket says 10.20.193.18:57080) 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b04d1d640 1 -- 10.20.193.18:0/3791264360 learned_addr learned my addr 10.20.193.18:0/3791264360 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.929+0000 7f5b05d1f640 1 --2- 10.20.193.18:0/3791264360 >> v2:10.20.193.18:3300/0 conn(0x7f5b000c8b00 0x7f5b000c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): get_auth_request con 0x7f5b000c8b00 auth_method 0 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:40.935 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.930+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 1 -- 10.20.193.18:0/3791264360 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58123102 0 0) 0x7f5b000c9490 con 0x7f5b00153d80 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 10 monclient(hunting): _negotiate 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 10 monclient(hunting): my global_id is 4294 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.931+0000 7f5aeffff640 1 -- 10.20.193.18:0/3791264360 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ae8003620 con 0x7f5b00153d80 2026-02-24T18:34:40.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_done global_id 4284 payload 274 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 1 -- 10.20.193.18:0/3791264360 >> v1:10.20.193.32:6789/0 conn(0x7f5b00153d80 legacy=0x7f5b00074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 1 -- 10.20.193.18:0/3791264360 >> v1:10.20.193.18:6789/0 conn(0x7f5b001539a0 legacy=0x7f5b000c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 10 monclient: _finish_hunting 0 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 1 monclient: found mon.noname-a 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.932+0000 7f5b05d1f640 1 -- 10.20.193.18:0/3791264360 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b000c5710 con 0x7f5b000c8b00 2026-02-24T18:34:40.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.935+0000 7f5b05d1f640 1 --2- 10.20.193.18:0/3791264360 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b000c8b00 0x7f5b000c8f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009d90 tx=0x7f5afc02f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc086917e84b0e9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:40.944 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 1 -- 10.20.193.18:0/3791264360 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc03c070 con 0x7f5b000c8b00 2026-02-24T18:34:40.944 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: dump: 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:40.945 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: _finish_auth 0 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: _check_auth_tickets 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 1 -- 10.20.193.18:0/3791264360 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc037440 con 0x7f5b000c8b00 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5b077a9640 5 monclient: authenticate success, global_id 4284 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5b077a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5b077a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 1 -- 10.20.193.18:0/3791264360 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc037810 con 0x7f5b000c8b00 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5b077a9640 4 set_mon_vals no callback set 2026-02-24T18:34:40.946 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.937+0000 7f5aeffff640 10 monclient: dump: 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:40.947 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 10 monclient: shutdown 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- 10.20.193.18:0/3791264360 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b000c8b00 msgr2=0x7f5b000c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 --2- 10.20.193.18:0/3791264360 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b000c8b00 0x7f5b000c8f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009d90 tx=0x7f5afc02f1e0 comp rx=0 tx=0).stop 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- 10.20.193.18:0/3791264360 shutdown_connections 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 --2- 10.20.193.18:0/3791264360 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b000c8b00 0x7f5b000c8f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- 10.20.193.18:0/3791264360 >> 10.20.193.18:0/3791264360 conn(0x7f5b000c9f40 msgr2=0x7f5b000ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- 10.20.193.18:0/3791264360 shutdown_connections 2026-02-24T18:34:40.948 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- 10.20.193.18:0/3791264360 wait complete. 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 10 monclient: build_initial_monmap 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 10 monclient: monmap: 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:40.949 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 librados: starting msgr at 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 librados: starting objecter 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 Processor -- start 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 -- start start 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 librados: setting wanted keys 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 1 librados: calling monclient init 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.938+0000 7f5b077a9640 10 monclient: init 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b000c8b00 0x7f5b0015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-c con 0x7f5b000c8b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:40.950 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b001539a0 0x7f5b001f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-b con 0x7f5b001539a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): picked mon.noname-a con 0x7f5b00153d80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5b000c8300 con 0x7f5b00153d80 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b000c8020 con 0x7f5b001539a0 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b000c85e0 con 0x7f5b000c8b00 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b077a9640 10 monclient(hunting): authenticate will time out at 1424.620261s 2026-02-24T18:34:40.951 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b04d1d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b001539a0 0x7f5b001f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50544/0 (socket says 10.20.193.18:50544) 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 1 -- 10.20.193.18:0/375082262 learned_addr learned my addr 10.20.193.18:0/375082262 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 10 monclient(hunting): get_auth_request con 0x7f5b00153d80 auth_method 0 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b05d1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b04d1d640 10 monclient(hunting): get_auth_request con 0x7f5b001539a0 auth_method 0 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b0551e640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b000c8b00 0x7f5b0015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b04d1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:40.952 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b04d1d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.939+0000 7f5b04d1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): get_auth_request con 0x7f5b000c8b00 auth_method 0 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b04d1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b04d1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b04d1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:40.953 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.940+0000 7f5b0551e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 10 monclient(hunting): handle_auth_done global_id 4290 payload 824 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 -- 10.20.193.18:0/375082262 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b000c8b00 msgr2=0x7f5b0015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b000c8b00 0x7f5b0015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 -- 10.20.193.18:0/375082262 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b001539a0 msgr2=0x7f5b001f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b001539a0 0x7f5b001f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 10 monclient: _finish_hunting 0 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 monclient: found mon.noname-a 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:34:40.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.942+0000 7f5b05d1f640 1 -- 10.20.193.18:0/375082262 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b00158f30 con 0x7f5b00153d80 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.943+0000 7f5b05d1f640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5afc03a040 tx=0x7f5afc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.943+0000 7f5aedffb640 10 client.?.objecter ms_handle_connect 0x7f5b00153d80 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.944+0000 7f5aedffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc03c050 con 0x7f5b00153d80 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: dump: 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:40.957 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: _finish_auth 0 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: _check_auth_tickets 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc048070 con 0x7f5b00153d80 2026-02-24T18:34:40.958 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc035a50 con 0x7f5b00153d80 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 5 monclient: authenticate success, global_id 4290 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _renew_subs 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0006d460 con 0x7f5b00153d80 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aef7fe640 4 set_mon_vals no callback set 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5aedffb640 10 monclient: dump: 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:40.959 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 client.4290.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _renew_subs 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0006da20 con 0x7f5b00153d80 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 1 librados: init done 2026-02-24T18:34:40.960 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:40.961 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:40.961 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:40.961 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.946+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b000c52d0 con 0x7f5b00153d80 2026-02-24T18:34:40.962 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.960+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f5afc035bf0 con 0x7f5b00153d80 2026-02-24T18:34:40.962 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.961+0000 7f5aedffb640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 0x7f5ad007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:40.963 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.961+0000 7f5b0551e640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 0x7f5ad007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:40.963 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.961+0000 7f5b0551e640 10 monclient: get_auth_request con 0x7f5ad0078dc0 auth_method 0 2026-02-24T18:34:40.963 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.962+0000 7f5aedffb640 10 client.4290.objecter ms_handle_connect 0x7f5ad0078dc0 2026-02-24T18:34:40.963 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:40.962+0000 7f5b0551e640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 0x7f5ad007b280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5af0002770 tx=0x7f5af0007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:41.039 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.037+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f5afc0845d0 con 0x7f5b00153d80 2026-02-24T18:34:41.039 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.037+0000 7f5aedffb640 10 client.4290.objecter ms_dispatch2 0x7f5b00000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:41.039 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.037+0000 7f5aedffb640 3 client.4290.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:41.040 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.037+0000 7f5aedffb640 3 client.4290.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:41.040 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.037+0000 7f5aedffb640 20 client.4290.objecter dump_active .. 0 homeless 2026-02-24T18:34:41.080 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.078+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5afc088590 con 0x7f5b00153d80 2026-02-24T18:34:41.080 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.078+0000 7f5aedffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:41.081 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.078+0000 7f5aedffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:41.170 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.169+0000 7f5b077a9640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:41.170 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.169+0000 7f5b077a9640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:41.171 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.169+0000 7f5b077a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:41.171 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.169+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b00070cf0 con 0x7f5b00153d80 2026-02-24T18:34:41.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.180+0000 7f5aedffb640 1 -- 10.20.193.18:0/375082262 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f5afc08c890 con 0x7f5b00153d80 2026-02-24T18:34:41.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.180+0000 7f5aedffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:41.182 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.180+0000 7f5aedffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:41.182 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:41.182 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:41.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 10 librados: watch_flush enter 2026-02-24T18:34:41.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 10 librados: watch_flush exit 2026-02-24T18:34:41.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 20 client.4290.objecter shutdown clearing up homeless session... 2026-02-24T18:34:41.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 10 client.4290.objecter successfully canceled tick 2026-02-24T18:34:41.186 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 msgr2=0x7f5ad007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 0x7f5ad007b280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5af0002770 tx=0x7f5af0007ef0 comp rx=0 tx=0).stop 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 10 monclient: shutdown 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 msgr2=0x7f5b0006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5afc03a040 tx=0x7f5afc004b40 comp rx=0 tx=0).stop 2026-02-24T18:34:41.187 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 shutdown_connections 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad0078dc0 0x7f5ad007b280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b001539a0 0x7f5b001f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b00153d80 0x7f5b0006e500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 --2- 10.20.193.18:0/375082262 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b000c8b00 0x7f5b0015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 >> 10.20.193.18:0/375082262 conn(0x7f5b000c9f40 msgr2=0x7f5b000ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:41.188 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 shutdown_connections 2026-02-24T18:34:41.189 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 -- 10.20.193.18:0/375082262 wait complete. 2026-02-24T18:34:41.189 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:41.182+0000 7f5b077a9640 1 librados: shutdown 2026-02-24T18:34:41.195 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:48.808 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 10 monclient: get_monmap_and_config 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 10 monclient: build_initial_monmap 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 10 monclient: monmap: 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:48.872973+0000 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:48.872973+0000 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:48.873 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 1 Processor -- start 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 1 -- start start 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.871+0000 7f308146f640 10 monclient: init 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient: _add_conns ranks=[5,2,1,0,4,3] 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f307c1539a0 0x7f307c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): picked mon.noname-c con 0x7f307c1539a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f307c153d80 addr v1:10.20.193.110:6789/0 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f307c0c8b00 addr v1:10.20.193.32:6789/0 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307c075070 con 0x7f307c0c8b00 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307c0c5710 con 0x7f307c153d80 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f307c0c5cd0 con 0x7f307c1539a0 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:48.874 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f308146f640 10 monclient(hunting): authenticate will time out at 1432.554231s 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f307b7fe640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f307c0c8b00 0x7f307c0c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:57084/0 (socket says 10.20.193.18:57084) 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.872+0000 7f307b7fe640 1 -- 10.20.193.18:0/2089389351 learned_addr learned my addr 10.20.193.18:0/2089389351 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.873+0000 7f307affd640 1 --2- 10.20.193.18:0/2089389351 >> v2:10.20.193.110:3300/0 conn(0x7f307c1539a0 0x7f307c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.873+0000 7f307affd640 10 monclient(hunting): get_auth_request con 0x7f307c1539a0 auth_method 0 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.873+0000 7f307affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.873+0000 7f307affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.875 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.873+0000 7f307affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822189478 0 0) 0x7f306c002ca0 con 0x7f307c0c8b00 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 10 monclient(hunting): my global_id is 4300 2026-02-24T18:34:48.876 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.874+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3068003620 con 0x7f307c0c8b00 2026-02-24T18:34:48.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.875+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.875+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.875+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:48.878 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.1 v1:10.20.193.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1234289193 0 0) 0x7f3068003620 con 0x7f307c0c8b00 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 >> v2:10.20.193.110:3300/0 conn(0x7f307c1539a0 msgr2=0x7f307c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 --2- 10.20.193.18:0/2089389351 >> v2:10.20.193.110:3300/0 conn(0x7f307c1539a0 0x7f307c0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 >> v1:10.20.193.110:6789/0 conn(0x7f307c153d80 legacy=0x7f307c074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _finish_hunting 0 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 monclient: found mon.noname-b-legacy 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.32:6789/0 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 --> v1:10.20.193.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c0c59f0 con 0x7f307c0c8b00 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _finish_auth 0 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:34:48.879 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.1 v1:10.20.193.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f306c003450 con 0x7f307c0c8b00 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: dump: 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:48.880 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] but i'm connected to v1:10.20.193.32:6789/0, reconnecting 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 >> v1:10.20.193.32:6789/0 conn(0x7f307c0c8b00 legacy=0x7f307c0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3068003bf0 0x7f3068004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): picked mon.c con 0x7f3068003bf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3068004900 0x7f3068004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): picked mon.b con 0x7f3068004900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 0x7f306800bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.881 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): picked mon.a con 0x7f30680054c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f307c0c5cd0 con 0x7f30680054c0 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f306800c3e0 con 0x7f3068004900 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f306800c5e0 con 0x7f3068003bf0 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:48.882 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f3079ffb640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:34:48.889 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f307b7fe640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 0x7f306800bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.889 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f307b7fe640 10 monclient(hunting): get_auth_request con 0x7f30680054c0 auth_method 0 2026-02-24T18:34:48.889 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f307b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.889 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f307b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.889 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.876+0000 7f307b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3068004900 0x7f3068004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): get_auth_request con 0x7f3068004900 auth_method 0 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.877+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 10 monclient(hunting): handle_auth_done global_id 4300 payload 293 2026-02-24T18:34:48.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 -- 10.20.193.18:0/2089389351 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3068003bf0 msgr2=0x7f3068004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3068003bf0 0x7f3068004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 -- 10.20.193.18:0/2089389351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3068004900 msgr2=0x7f3068004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3068004900 0x7f3068004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 monclient: found mon.a 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 -- 10.20.193.18:0/2089389351 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3068003620 con 0x7f30680054c0 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:48.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f308146f640 5 monclient: authenticate success, global_id 4300 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f308146f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.878+0000 7f307b7fe640 1 -- 10.20.193.18:0/2089389351 reap_dead start 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.879+0000 7f307b7fe640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 0x7f306800bb00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f306c00a9a0 tx=0x7f306c02fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25edf557bbb587 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00a5c0 con 0x7f30680054c0 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: dump: 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:48.892 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c034030 con 0x7f30680054c0 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 1 -- 10.20.193.18:0/2089389351 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c030a30 con 0x7f30680054c0 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 4 set_mon_vals no callback set 2026-02-24T18:34:48.893 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f3079ffb640 10 monclient: dump: 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 10 monclient: shutdown 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 -- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 msgr2=0x7f306800bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 0x7f306800bb00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f306c00a9a0 tx=0x7f306c02fd70 comp rx=0 tx=0).stop 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 -- 10.20.193.18:0/2089389351 shutdown_connections 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 --2- 10.20.193.18:0/2089389351 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f30680054c0 0x7f306800bb00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 -- 10.20.193.18:0/2089389351 >> 10.20.193.18:0/2089389351 conn(0x7f307c0c9f40 msgr2=0x7f307c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 -- 10.20.193.18:0/2089389351 shutdown_connections 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.881+0000 7f308146f640 1 -- 10.20.193.18:0/2089389351 wait complete. 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient: build_initial_monmap 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient: monmap: 2026-02-24T18:34:48.895 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 librados: starting msgr at 2026-02-24T18:34:48.896 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 librados: starting objecter 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 Processor -- start 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 -- start start 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 librados: setting wanted keys 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 librados: calling monclient init 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient: init 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f30680054c0 0x7f307c1e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): picked mon.noname-c con 0x7f30680054c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): picked mon.noname-a con 0x7f307c1e7b80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:48.897 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f307c142110 0x7f307c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): picked mon.noname-b con 0x7f307c142110 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f307c0c85e0 con 0x7f307c1e7b80 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f307c0c8300 con 0x7f307c142110 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f307c0c52d0 con 0x7f30680054c0 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.882+0000 7f308146f640 10 monclient(hunting): authenticate will time out at 1432.564816s 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f307c142110 0x7f307c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.898 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f30680054c0 0x7f307c1e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50560/0 (socket says 10.20.193.18:50560) 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 1 -- 10.20.193.18:0/2667208471 learned_addr learned my addr 10.20.193.18:0/2667208471 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): get_auth_request con 0x7f307c1e7b80 auth_method 0 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): get_auth_request con 0x7f307c142110 auth_method 0 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): get_auth_request con 0x7f30680054c0 auth_method 0 2026-02-24T18:34:48.899 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.883+0000 7f307affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 10 monclient(hunting): handle_auth_done global_id 4293 payload 824 2026-02-24T18:34:48.900 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 -- 10.20.193.18:0/2667208471 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f30680054c0 msgr2=0x7f307c1e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f30680054c0 0x7f307c1e7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 -- 10.20.193.18:0/2667208471 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f307c142110 msgr2=0x7f307c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f307c142110 0x7f307c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 monclient: found mon.noname-a 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.884+0000 7f307a7fc640 1 -- 10.20.193.18:0/2667208471 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c142590 con 0x7f307c1e7b80 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.885+0000 7f307a7fc640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f306400e700 tx=0x7f306400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.885+0000 7f305b7fe640 10 client.?.objecter ms_handle_connect 0x7f307c1e7b80 2026-02-24T18:34:48.901 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.885+0000 7f305b7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:48.903 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640068d0 con 0x7f307c1e7b80 2026-02-24T18:34:48.903 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:48.903 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: dump: 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:34:48.904 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064019070 con 0x7f307c1e7b80 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064006cd0 con 0x7f307c1e7b80 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 5 monclient: authenticate success, global_id 4293 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 10 monclient: _renew_subs 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f30797fa640 4 set_mon_vals no callback set 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c0cc7f0 con 0x7f307c1e7b80 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f305b7fe640 10 monclient: dump: 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.905 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 10 client.4293.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 10 monclient: _renew_subs 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c0ccdb0 con 0x7f307c1e7b80 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.887+0000 7f308146f640 1 librados: init done 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.888+0000 7f308146f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.888+0000 7f308146f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.888+0000 7f308146f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.888+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307c0c8f50 con 0x7f307c1e7b80 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.900+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f306400c660 con 0x7f307c1e7b80 2026-02-24T18:34:48.906 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.901+0000 7f305b7fe640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 0x7f304807b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.901+0000 7f307affd640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 0x7f304807b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.901+0000 7f307affd640 10 monclient: get_auth_request con 0x7f3048078d70 auth_method 0 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.901+0000 7f307affd640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 0x7f304807b230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30700094d0 tx=0x7f30700095a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.901+0000 7f305b7fe640 10 client.4293.objecter ms_handle_connect 0x7f3048078d70 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.903+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f306405e450 con 0x7f307c1e7b80 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.903+0000 7f305b7fe640 10 client.4293.objecter ms_dispatch2 0x7f307c000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:48.907 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.903+0000 7f305b7fe640 3 client.4293.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:48.908 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.903+0000 7f305b7fe640 3 client.4293.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:48.908 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:48.903+0000 7f305b7fe640 20 client.4293.objecter dump_active .. 0 homeless 2026-02-24T18:34:49.014 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.012+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f307c142590 con 0x7f307c1e7b80 2026-02-24T18:34:49.015 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.012+0000 7f305b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:49.015 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.012+0000 7f305b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:49.105 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.103+0000 7f308146f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:49.106 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.103+0000 7f308146f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:49.106 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.103+0000 7f308146f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:34:49.106 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.103+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f307c1477f0 con 0x7f307c1e7b80 2026-02-24T18:34:49.116 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.114+0000 7f305b7fe640 1 -- 10.20.193.18:0/2667208471 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f3064062410 con 0x7f307c1e7b80 2026-02-24T18:34:49.116 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.114+0000 7f305b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:49.116 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.114+0000 7f305b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:49.117 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:49.117 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:49.119 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 10 librados: watch_flush enter 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 10 librados: watch_flush exit 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 20 client.4293.objecter shutdown clearing up homeless session... 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 10 client.4293.objecter successfully canceled tick 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 msgr2=0x7f304807b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 0x7f304807b230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30700094d0 tx=0x7f30700095a0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 10 monclient: shutdown 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 msgr2=0x7f307c141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f306400e700 tx=0x7f306400ebd0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 shutdown_connections 2026-02-24T18:34:49.120 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3048078d70 0x7f304807b230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f307c142110 0x7f307c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f307c1e7b80 0x7f307c141bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 --2- 10.20.193.18:0/2667208471 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f30680054c0 0x7f307c1e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 >> 10.20.193.18:0/2667208471 conn(0x7f307c0c9f40 msgr2=0x7f307c0c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 shutdown_connections 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 -- 10.20.193.18:0/2667208471 wait complete. 2026-02-24T18:34:49.121 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:49.116+0000 7f308146f640 1 librados: shutdown 2026-02-24T18:34:49.129 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:56.740 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:34:56.803 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.801+0000 7feba5851640 10 monclient: get_monmap_and_config 2026-02-24T18:34:56.803 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.801+0000 7feba5851640 10 monclient: build_initial_monmap 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.801+0000 7feba5851640 10 monclient: monmap: 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:34:56.802753+0000 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:34:56.802753+0000 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:34:56.804 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:34:56.805 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:34:56.805 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.802+0000 7feba5851640 1 Processor -- start 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.802+0000 7feba5851640 1 -- start start 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.802+0000 7feba5851640 10 monclient: init 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient: _add_conns ranks=[3,0,4,2,1,5] 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7feba0156000 0x7feba00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): picked mon.noname-a con 0x7feba0156000 addr v2:10.20.193.18:3300/0 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feba006e6c0 addr v1:10.20.193.18:6789/0 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7feba015d250 0x7feba015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.808 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): picked mon.noname-b con 0x7feba015d250 addr v2:10.20.193.32:3300/0 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba00cc9a0 con 0x7feba006e6c0 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7feba00cd890 con 0x7feba0156000 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7feba00cc740 con 0x7feba015d250 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5851640 10 monclient(hunting): authenticate will time out at 1440.484117s 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7feba015d250 0x7feba015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba484f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7feba0156000 0x7feba00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7feba015d250 0x7feba015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:56202/0 (socket says 10.20.193.18:56202) 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 1 -- 10.20.193.18:0/1663867026 learned_addr learned my addr 10.20.193.18:0/1663867026 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:56.809 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 10 monclient(hunting): get_auth_request con 0x7feba015d250 auth_method 0 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.803+0000 7feba5050640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba484f640 10 monclient(hunting): get_auth_request con 0x7feba0156000 auth_method 0 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba484f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba484f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba484f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.804+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:56.810 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 10 monclient(hunting): handle_auth_done global_id 4306 payload 274 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 1 -- 10.20.193.18:0/1663867026 >> v2:10.20.193.18:3300/0 conn(0x7feba0156000 msgr2=0x7feba00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 1 --2- 10.20.193.18:0/1663867026 >> v2:10.20.193.18:3300/0 conn(0x7feba0156000 0x7feba00cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 1 -- 10.20.193.18:0/1663867026 >> v1:10.20.193.18:6789/0 conn(0x7feba006e6c0 legacy=0x7feba00cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 10 monclient: _finish_hunting 0 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 1 monclient: found mon.noname-b 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.805+0000 7feba5050640 1 -- 10.20.193.18:0/1663867026 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00ccc80 con 0x7feba015d250 2026-02-24T18:34:56.811 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.806+0000 7feba5050640 1 --2- 10.20.193.18:0/1663867026 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba015d250 0x7feba015d690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feba008e6c0 tx=0x7feb9c02f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5365958696e26885 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:56.817 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 1 -- 10.20.193.18:0/1663867026 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02fd70 con 0x7feba015d250 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: dump: 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:56.818 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: _finish_auth 0 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: _check_auth_tickets 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 5 monclient: authenticate success, global_id 4306 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 1 -- 10.20.193.18:0/1663867026 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c0373f0 con 0x7feba015d250 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 1 -- 10.20.193.18:0/1663867026 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0377c0 con 0x7feba015d250 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:56.819 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feb977fe640 10 monclient: dump: 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.820 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 10 monclient: get_monmap_and_config success 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 4 set_mon_vals no callback set 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 10 monclient: shutdown 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 -- 10.20.193.18:0/1663867026 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba015d250 msgr2=0x7feba015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 --2- 10.20.193.18:0/1663867026 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba015d250 0x7feba015d690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feba008e6c0 tx=0x7feb9c02f270 comp rx=0 tx=0).stop 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 -- 10.20.193.18:0/1663867026 shutdown_connections 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 --2- 10.20.193.18:0/1663867026 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba015d250 0x7feba015d690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 --2- 10.20.193.18:0/1663867026 >> v2:10.20.193.18:3300/0 conn(0x7feba0156000 0x7feba00cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 -- 10.20.193.18:0/1663867026 >> 10.20.193.18:0/1663867026 conn(0x7feba006d9e0 msgr2=0x7feba006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 -- 10.20.193.18:0/1663867026 shutdown_connections 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.808+0000 7feba5851640 1 -- 10.20.193.18:0/1663867026 wait complete. 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 10 monclient: build_initial_monmap 2026-02-24T18:34:56.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 10 monclient: monmap: 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 librados: starting msgr at 2026-02-24T18:34:56.822 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 librados: starting objecter 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 Processor -- start 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 -- start start 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 librados: setting wanted keys 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 1 librados: calling monclient init 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.809+0000 7feba5851640 10 monclient: init 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient: _reopen_session rank -1 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): picked mon.noname-b con 0x7feba006e6c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7feba0156000 0x7feba0147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): picked mon.noname-c con 0x7feba0156000 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:34:56.823 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7feba015d250 0x7feba01e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): picked mon.noname-a con 0x7feba015d250 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7feba00cd520 con 0x7feba015d250 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7feba00cd240 con 0x7feba006e6c0 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): start opening mon connection 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7feba01563e0 con 0x7feba0156000 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): _renew_subs 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5851640 10 monclient(hunting): authenticate will time out at 1440.491222s 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5050640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7feba015d250 0x7feba01e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.824 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:56218/0 (socket says 10.20.193.18:56218) 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 1 -- 10.20.193.18:0/2012724447 learned_addr learned my addr 10.20.193.18:0/2012724447 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): get_auth_request con 0x7feba006e6c0 auth_method 0 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5050640 10 monclient(hunting): get_auth_request con 0x7feba015d250 auth_method 0 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5050640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5050640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba5050640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feb97fff640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7feba0156000 0x7feba0147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:56.825 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.810+0000 7feba484f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feba5050640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): get_auth_request con 0x7feba0156000 auth_method 0 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.811+0000 7feb97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 10 monclient(hunting): handle_auth_done global_id 4309 payload 824 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 -- 10.20.193.18:0/2012724447 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7feba0156000 msgr2=0x7feba0147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:56.826 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7feba0156000 0x7feba0147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 -- 10.20.193.18:0/2012724447 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7feba015d250 msgr2=0x7feba01e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7feba015d250 0x7feba01e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 10 monclient: _finish_hunting 0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 monclient: found mon.noname-b 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.812+0000 7feba484f640 1 -- 10.20.193.18:0/2012724447 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00d1050 con 0x7feba006e6c0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.813+0000 7feba484f640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb9000ec10 tx=0x7feb9000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.813+0000 7feb957fa640 10 client.?.objecter ms_handle_connect 0x7feba006e6c0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.813+0000 7feb957fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90017070 con 0x7feba006e6c0 2026-02-24T18:34:56.827 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: dump: 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:56.828 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: _finish_auth 0 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: _check_auth_tickets 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 5 monclient: authenticate success, global_id 4309 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _renew_subs 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba00cf790 con 0x7feba006e6c0 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 client.4309.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _renew_subs 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba00cfd50 con 0x7feba006e6c0 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 1 librados: init done 2026-02-24T18:34:56.829 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9000cc40 con 0x7feba006e6c0 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90021060 con 0x7feba006e6c0 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb96ffd640 4 set_mon_vals no callback set 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feb957fa640 10 monclient: dump: 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:34:56.830 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:56.831 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.815+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba00cc200 con 0x7feba006e6c0 2026-02-24T18:34:56.927 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.926+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7feb90024060 con 0x7feba006e6c0 2026-02-24T18:34:56.928 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.926+0000 7feb957fa640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 0x7feb7007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:34:56.929 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.927+0000 7feb97fff640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 0x7feb7007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:34:56.929 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.927+0000 7feb97fff640 10 monclient: get_auth_request con 0x7feb70078d70 auth_method 0 2026-02-24T18:34:56.929 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.928+0000 7feb97fff640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 0x7feb7007b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feba00cb8e0 tx=0x7feb88007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:34:56.929 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.928+0000 7feb957fa640 10 client.4309.objecter ms_handle_connect 0x7feb70078d70 2026-02-24T18:34:56.936 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7feb9005de70 con 0x7feba006e6c0 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 10 client.4309.objecter ms_dispatch2 0x7feba0000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 3 client.4309.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 3 client.4309.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 20 client.4309.objecter dump_active .. 0 homeless 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feb90061e30 con 0x7feba006e6c0 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:34:56.937 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:56.935+0000 7feb957fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:34:57.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.025+0000 7feba5851640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:57.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.025+0000 7feba5851640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:57.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.025+0000 7feba5851640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:34:57.027 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.025+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feba00cbc90 con 0x7feba006e6c0 2026-02-24T18:34:57.037 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.036+0000 7feb957fa640 1 -- 10.20.193.18:0/2012724447 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7feb90066130 con 0x7feba006e6c0 2026-02-24T18:34:57.037 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.036+0000 7feb957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:34:57.037 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.036+0000 7feb957fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:34:57.038 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:34:57.038 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:34:57.040 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 10 librados: watch_flush enter 2026-02-24T18:34:57.040 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 10 librados: watch_flush exit 2026-02-24T18:34:57.040 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 20 client.4309.objecter shutdown clearing up homeless session... 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 10 client.4309.objecter successfully canceled tick 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 msgr2=0x7feb7007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 0x7feb7007b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feba00cb8e0 tx=0x7feb88007f30 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 10 monclient: shutdown 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 msgr2=0x7feba00d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb9000ec10 tx=0x7feb9000c520 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 shutdown_connections 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feb70078d70 0x7feb7007b230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7feba015d250 0x7feba01e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7feba0156000 0x7feba0147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 --2- 10.20.193.18:0/2012724447 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7feba006e6c0 0x7feba00d0b10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 >> 10.20.193.18:0/2012724447 conn(0x7feba006d9e0 msgr2=0x7feba00ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 shutdown_connections 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 -- 10.20.193.18:0/2012724447 wait complete. 2026-02-24T18:34:57.041 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:34:57.039+0000 7feba5851640 1 librados: shutdown 2026-02-24T18:34:57.049 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:34:59.073 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:37.018 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.075 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:37.021 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.076 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:37.022 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.077 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:36.985 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.079 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:37.025 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.079 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:36.988 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.094 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:37.040 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.097 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:37.006 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:34:59.098 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:37.044 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:04.664 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:04.726 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.724+0000 7f9b9a4da640 10 monclient: get_monmap_and_config 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.724+0000 7f9b9a4da640 10 monclient: build_initial_monmap 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.724+0000 7f9b9a4da640 10 monclient: monmap: 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:35:04.725697+0000 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:35:04.725697+0000 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:04.727 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 Processor -- start 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 -- start start 2026-02-24T18:35:04.728 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient: init 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient: _add_conns ranks=[2,0,3,1,5,4] 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9b941539a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b94153d80 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f9b940c8b00 0x7f9b940c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-a con 0x7f9b940c8b00 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94075070 con 0x7f9b94153d80 2026-02-24T18:35:04.731 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b940c9490 con 0x7f9b941539a0 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9b940c5cd0 con 0x7f9b940c8b00 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.725+0000 7f9b9a4da640 10 monclient(hunting): authenticate will time out at 1448.406867s 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b98a50640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f9b940c8b00 0x7f9b940c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b93fff640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f9b941539a0 0x7f9b940c7270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:54528/0 (socket says 10.20.193.18:54528) 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b93fff640 1 -- 10.20.193.18:0/2442276953 learned_addr learned my addr 10.20.193.18:0/2442276953 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b98a50640 10 monclient(hunting): get_auth_request con 0x7f9b940c8b00 auth_method 0 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b98a50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b98a50640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.726+0000 7f9b98a50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:04.732 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.727+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.727+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.727+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 1 -- 10.20.193.18:0/2442276953 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394721320 0 0) 0x7f9b940c9490 con 0x7f9b941539a0 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 10 monclient(hunting): my global_id is 4307 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b92ffd640 1 -- 10.20.193.18:0/2442276953 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b78003620 con 0x7f9b941539a0 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 10 monclient(hunting): handle_auth_done global_id 4305 payload 274 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 1 -- 10.20.193.18:0/2442276953 >> v1:10.20.193.110:6789/0 conn(0x7f9b941539a0 legacy=0x7f9b940c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 1 -- 10.20.193.18:0/2442276953 >> v1:10.20.193.18:6789/0 conn(0x7f9b94153d80 legacy=0x7f9b94074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.733 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 10 monclient: _finish_hunting 0 2026-02-24T18:35:04.734 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 1 monclient: found mon.noname-a 2026-02-24T18:35:04.734 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:04.734 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.734 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.728+0000 7f9b98a50640 1 -- 10.20.193.18:0/2442276953 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b940c5710 con 0x7f9b940c8b00 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.730+0000 7f9b98a50640 1 --2- 10.20.193.18:0/2442276953 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b940c8f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009d90 tx=0x7f9b8802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acac30f086d3f554 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 1 -- 10.20.193.18:0/2442276953 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8803c070 con 0x7f9b940c8b00 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: dump: 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.740 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 1 -- 10.20.193.18:0/2442276953 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b88037440 con 0x7f9b940c8b00 2026-02-24T18:35:04.741 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 1 -- 10.20.193.18:0/2442276953 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88037810 con 0x7f9b940c8b00 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b9a4da640 5 monclient: authenticate success, global_id 4305 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b9a4da640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b9a4da640 4 set_mon_vals no callback set 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.732+0000 7f9b92ffd640 10 monclient: dump: 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.742 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 10 monclient: shutdown 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 -- 10.20.193.18:0/2442276953 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 msgr2=0x7f9b940c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/2442276953 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b940c8f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009d90 tx=0x7f9b8802f1e0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 -- 10.20.193.18:0/2442276953 shutdown_connections 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/2442276953 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b940c8f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.743 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 -- 10.20.193.18:0/2442276953 >> 10.20.193.18:0/2442276953 conn(0x7f9b940c9f40 msgr2=0x7f9b940ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 -- 10.20.193.18:0/2442276953 shutdown_connections 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.733+0000 7f9b9a4da640 1 -- 10.20.193.18:0/2442276953 wait complete. 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient: build_initial_monmap 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient: monmap: 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:04.744 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 librados: starting msgr at 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 librados: starting objecter 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 Processor -- start 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 -- start start 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 librados: setting wanted keys 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 librados: calling monclient init 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient: init 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:04.745 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-a con 0x7f9b940c8b00 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b941539a0 0x7f9b941f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-b con 0x7f9b941539a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b94153d80 0x7f9b94154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): picked mon.noname-c con 0x7f9b94153d80 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940c8300 con 0x7f9b940c8b00 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940c8020 con 0x7f9b941539a0 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940c85e0 con 0x7f9b94153d80 2026-02-24T18:35:04.746 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.734+0000 7f9b9a4da640 10 monclient(hunting): authenticate will time out at 1448.415956s 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b94153d80 0x7f9b94154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:43208/0 (socket says 10.20.193.18:43208) 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 1 -- 10.20.193.18:0/104403849 learned_addr learned my addr 10.20.193.18:0/104403849 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): get_auth_request con 0x7f9b940c8b00 auth_method 0 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b937fe640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b941539a0 0x7f9b941f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): get_auth_request con 0x7f9b94153d80 auth_method 0 2026-02-24T18:35:04.747 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b937fe640 10 monclient(hunting): get_auth_request con 0x7f9b941539a0 auth_method 0 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b937fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:04.748 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.735+0000 7f9b98a50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.736+0000 7f9b937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.736+0000 7f9b937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.736+0000 7f9b937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 10 monclient(hunting): handle_auth_done global_id 4311 payload 824 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 -- 10.20.193.18:0/104403849 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b94153d80 msgr2=0x7f9b94154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b94153d80 0x7f9b94154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 -- 10.20.193.18:0/104403849 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b941539a0 msgr2=0x7f9b941f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b941539a0 0x7f9b941f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 monclient: found mon.noname-a 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.749 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.737+0000 7f9b93fff640 1 -- 10.20.193.18:0/104403849 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941572c0 con 0x7f9b940c8b00 2026-02-24T18:35:04.750 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.738+0000 7f9b93fff640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c700 tx=0x7f9b8000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:04.750 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.738+0000 7f9b90ff9640 10 client.?.objecter ms_handle_connect 0x7f9b940c8b00 2026-02-24T18:35:04.750 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.738+0000 7f9b90ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80017070 con 0x7f9b940c8b00 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: dump: 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:04.752 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80005bf0 con 0x7f9b940c8b00 2026-02-24T18:35:04.753 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80007070 con 0x7f9b940c8b00 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b90ff9640 10 monclient: dump: 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:04.754 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b927fc640 4 set_mon_vals no callback set 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 5 monclient: authenticate success, global_id 4311 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 10 monclient: _renew_subs 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9406d560 con 0x7f9b940c8b00 2026-02-24T18:35:04.755 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 10 client.4311.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 10 monclient: _renew_subs 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9406db20 con 0x7f9b940c8b00 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.739+0000 7f9b9a4da640 1 librados: init done 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.740+0000 7f9b9a4da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.740+0000 7f9b9a4da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.740+0000 7f9b9a4da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.756 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.740+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b940c52d0 con 0x7f9b940c8b00 2026-02-24T18:35:04.757 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.751+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f9b80007210 con 0x7f9b940c8b00 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.752+0000 7f9b90ff9640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 0x7f9b6407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.752+0000 7f9b937fe640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 0x7f9b6407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.752+0000 7f9b937fe640 10 monclient: get_auth_request con 0x7f9b64078df0 auth_method 0 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.752+0000 7f9b90ff9640 10 client.4311.objecter ms_handle_connect 0x7f9b64078df0 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.752+0000 7f9b937fe640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 0x7f9b6407b2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9b9415e000 tx=0x7f9b84007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.754+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f9b8005d440 con 0x7f9b940c8b00 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.754+0000 7f9b90ff9640 10 client.4311.objecter ms_dispatch2 0x7f9b94000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.754+0000 7f9b90ff9640 3 client.4311.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.754+0000 7f9b90ff9640 3 client.4311.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:35:04.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.754+0000 7f9b90ff9640 20 client.4311.objecter dump_active .. 0 homeless 2026-02-24T18:35:04.864 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.862+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9b80021020 con 0x7f9b940c8b00 2026-02-24T18:35:04.864 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.862+0000 7f9b90ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:04.864 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.862+0000 7f9b90ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:04.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.952+0000 7f9b9a4da640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:04.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.952+0000 7f9b9a4da640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:04.954 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.952+0000 7f9b9a4da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:04.955 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.952+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b9406bab0 con 0x7f9b940c8b00 2026-02-24T18:35:04.965 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.963+0000 7f9b90ff9640 1 -- 10.20.193.18:0/104403849 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f9b80061400 con 0x7f9b940c8b00 2026-02-24T18:35:04.965 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.963+0000 7f9b90ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:04.965 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.963+0000 7f9b90ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:04.965 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:35:04.965 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:04.967 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.965+0000 7f9b9a4da640 10 librados: watch_flush enter 2026-02-24T18:35:04.967 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.965+0000 7f9b9a4da640 10 librados: watch_flush exit 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.965+0000 7f9b9a4da640 20 client.4311.objecter shutdown clearing up homeless session... 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.965+0000 7f9b9a4da640 10 client.4311.objecter successfully canceled tick 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 msgr2=0x7f9b6407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 0x7f9b6407b2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9b9415e000 tx=0x7f9b84007ef0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 10 monclient: shutdown 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 msgr2=0x7f9b94156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c700 tx=0x7f9b8000cbd0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 shutdown_connections 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b64078df0 0x7f9b6407b2b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b941539a0 0x7f9b941f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b94153d80 0x7f9b94154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 --2- 10.20.193.18:0/104403849 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b940c8b00 0x7f9b94156d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 >> 10.20.193.18:0/104403849 conn(0x7f9b940c9f40 msgr2=0x7f9b940ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 shutdown_connections 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 -- 10.20.193.18:0/104403849 wait complete. 2026-02-24T18:35:04.968 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:04.966+0000 7f9b9a4da640 1 librados: shutdown 2026-02-24T18:35:04.976 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:35:12.591 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 10 monclient: get_monmap_and_config 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 10 monclient: build_initial_monmap 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 10 monclient: monmap: 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:35:12.653006+0000 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:35:12.653006+0000 2026-02-24T18:35:12.653 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:12.654 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.657 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 1 Processor -- start 2026-02-24T18:35:12.657 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 1 -- start start 2026-02-24T18:35:12.657 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.652+0000 7f2b56843640 10 monclient: init 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient: _add_conns ranks=[3,0,4,2,1,5] 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2b501539a0 0x7f2b500c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-a con 0x7f2b501539a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2b50153d80 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2b500c8b00 0x7f2b500c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-b con 0x7f2b500c8b00 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b50075070 con 0x7f2b50153d80 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2b500c5cd0 con 0x7f2b501539a0 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:12.658 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2b500c9490 con 0x7f2b500c8b00 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b56843640 10 monclient(hunting): authenticate will time out at 1456.334152s 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b4ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2b501539a0 0x7f2b500c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.653+0000 7f2b54db9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2b500c8b00 0x7f2b500c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2b500c8b00 0x7f2b500c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:48392/0 (socket says 10.20.193.18:48392) 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 1 -- 10.20.193.18:0/3707821618 learned_addr learned my addr 10.20.193.18:0/3707821618 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): get_auth_request con 0x7f2b500c8b00 auth_method 0 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:12.659 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): get_auth_request con 0x7f2b501539a0 auth_method 0 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.654+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 10 monclient(hunting): handle_auth_done global_id 4315 payload 274 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 1 -- 10.20.193.18:0/3707821618 >> v2:10.20.193.18:3300/0 conn(0x7f2b501539a0 msgr2=0x7f2b500c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:12.660 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 1 --2- 10.20.193.18:0/3707821618 >> v2:10.20.193.18:3300/0 conn(0x7f2b501539a0 0x7f2b500c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 1 -- 10.20.193.18:0/3707821618 >> v1:10.20.193.18:6789/0 conn(0x7f2b50153d80 legacy=0x7f2b50074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 1 monclient: found mon.noname-b 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.661 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.655+0000 7f2b54db9640 1 -- 10.20.193.18:0/3707821618 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b500c5710 con 0x7f2b500c8b00 2026-02-24T18:35:12.666 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.656+0000 7f2b54db9640 1 --2- 10.20.193.18:0/3707821618 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b500c8b00 0x7f2b500c8f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b50070f60 tx=0x7f2b4402f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564ce56c4ba235fc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:12.666 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 1 -- 10.20.193.18:0/3707821618 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402fcf0 con 0x7f2b500c8b00 2026-02-24T18:35:12.666 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: dump: 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:12.667 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: _finish_auth 0 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 5 monclient: authenticate success, global_id 4315 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 1 -- 10.20.193.18:0/3707821618 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b440373f0 con 0x7f2b500c8b00 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 1 -- 10.20.193.18:0/3707821618 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b440377c0 con 0x7f2b500c8b00 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 4 set_mon_vals no callback set 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:12.668 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b4effd640 10 monclient: dump: 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 10 monclient: shutdown 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:12.669 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 -- 10.20.193.18:0/3707821618 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b500c8b00 msgr2=0x7f2b500c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 --2- 10.20.193.18:0/3707821618 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b500c8b00 0x7f2b500c8f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b50070f60 tx=0x7f2b4402f230 comp rx=0 tx=0).stop 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 -- 10.20.193.18:0/3707821618 shutdown_connections 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 --2- 10.20.193.18:0/3707821618 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b500c8b00 0x7f2b500c8f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 --2- 10.20.193.18:0/3707821618 >> v2:10.20.193.18:3300/0 conn(0x7f2b501539a0 0x7f2b500c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 -- 10.20.193.18:0/3707821618 >> 10.20.193.18:0/3707821618 conn(0x7f2b500c9f40 msgr2=0x7f2b500ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 -- 10.20.193.18:0/3707821618 shutdown_connections 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 -- 10.20.193.18:0/3707821618 wait complete. 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 10 monclient: build_initial_monmap 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 10 monclient: monmap: 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:35:12.670 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 librados: starting msgr at 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.659+0000 7f2b56843640 1 librados: starting objecter 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 Processor -- start 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 -- start start 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 librados: setting wanted keys 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 librados: calling monclient init 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient: init 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:12.671 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b500c8b00 0x7f2b501f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-c con 0x7f2b500c8b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-b con 0x7f2b501539a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b50153d80 0x7f2b50157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): picked mon.noname-a con 0x7f2b50153d80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500c8300 con 0x7f2b50153d80 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500c8020 con 0x7f2b501539a0 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500c85e0 con 0x7f2b500c8b00 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b56843640 10 monclient(hunting): authenticate will time out at 1456.341507s 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b54db9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b50153d80 0x7f2b50157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.672 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:48406/0 (socket says 10.20.193.18:48406) 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 1 -- 10.20.193.18:0/2097059837 learned_addr learned my addr 10.20.193.18:0/2097059837 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 10 monclient(hunting): get_auth_request con 0x7f2b501539a0 auth_method 0 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.660+0000 7f2b4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b500c8b00 0x7f2b501f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): get_auth_request con 0x7f2b50153d80 auth_method 0 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:12.673 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): get_auth_request con 0x7f2b500c8b00 auth_method 0 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b54db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.661+0000 7f2b4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 10 monclient(hunting): handle_auth_done global_id 4318 payload 824 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 -- 10.20.193.18:0/2097059837 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b500c8b00 msgr2=0x7f2b501f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b500c8b00 0x7f2b501f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 -- 10.20.193.18:0/2097059837 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b50153d80 msgr2=0x7f2b50157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b50153d80 0x7f2b50157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 monclient: found mon.noname-b 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:12.674 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.675 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.663+0000 7f2b4f7fe640 1 -- 10.20.193.18:0/2097059837 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5014eca0 con 0x7f2b501539a0 2026-02-24T18:35:12.675 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.664+0000 7f2b4f7fe640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c450 tx=0x7f2b4000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:12.675 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.664+0000 7f2b4cff9640 10 client.?.objecter ms_handle_connect 0x7f2b501539a0 2026-02-24T18:35:12.675 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.664+0000 7f2b4cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400052d0 con 0x7f2b501539a0 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 10 monclient: dump: 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:12.676 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b4cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 5 monclient: authenticate success, global_id 4318 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 10 monclient: _renew_subs 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b501581b0 con 0x7f2b501539a0 2026-02-24T18:35:12.677 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 10 client.4318.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 10 monclient: _renew_subs 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b50158770 con 0x7f2b501539a0 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.665+0000 7f2b56843640 1 librados: init done 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40005470 con 0x7f2b501539a0 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40006ad0 con 0x7f2b501539a0 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4cff9640 10 monclient: dump: 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:12.678 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b4e7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b56843640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b56843640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b56843640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.666+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b500c52d0 con 0x7f2b501539a0 2026-02-24T18:35:12.679 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.678+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f2b40023070 con 0x7f2b501539a0 2026-02-24T18:35:12.680 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.679+0000 7f2b4cff9640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 0x7f2b2007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:12.680 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.679+0000 7f2b4ffff640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 0x7f2b2007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:12.680 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.679+0000 7f2b4ffff640 10 monclient: get_auth_request con 0x7f2b20078d70 auth_method 0 2026-02-24T18:35:12.681 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.679+0000 7f2b4cff9640 10 client.4318.objecter ms_handle_connect 0x7f2b20078d70 2026-02-24T18:35:12.681 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.679+0000 7f2b4ffff640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 0x7f2b2007b230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b501f1640 tx=0x7f2b3c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:12.683 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.681+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f2b4005ddc0 con 0x7f2b501539a0 2026-02-24T18:35:12.683 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.681+0000 7f2b4cff9640 10 client.4318.objecter ms_dispatch2 0x7f2b50000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:35:12.683 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.681+0000 7f2b4cff9640 3 client.4318.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:35:12.683 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.681+0000 7f2b4cff9640 3 client.4318.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:35:12.683 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.681+0000 7f2b4cff9640 20 client.4318.objecter dump_active .. 0 homeless 2026-02-24T18:35:12.788 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.787+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2b40020020 con 0x7f2b501539a0 2026-02-24T18:35:12.789 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.787+0000 7f2b4cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:12.789 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.787+0000 7f2b4cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:12.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.875+0000 7f2b56843640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:12.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.875+0000 7f2b56843640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:12.877 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.875+0000 7f2b56843640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:12.878 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.875+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b500c9670 con 0x7f2b501539a0 2026-02-24T18:35:12.890 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.889+0000 7f2b4cff9640 1 -- 10.20.193.18:0/2097059837 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f2b5014eca0 con 0x7f2b501539a0 2026-02-24T18:35:12.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.889+0000 7f2b4cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:12.891 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.889+0000 7f2b4cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:12.891 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:35:12.891 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 10 librados: watch_flush enter 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 10 librados: watch_flush exit 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 20 client.4318.objecter shutdown clearing up homeless session... 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 10 client.4318.objecter successfully canceled tick 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 msgr2=0x7f2b2007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 0x7f2b2007b230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b501f1640 tx=0x7f2b3c009290 comp rx=0 tx=0).stop 2026-02-24T18:35:12.894 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 10 monclient: shutdown 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 msgr2=0x7f2b5014e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c450 tx=0x7f2b4000c920 comp rx=0 tx=0).stop 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 shutdown_connections 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2b20078d70 0x7f2b2007b230 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b501539a0 0x7f2b5014e760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b50153d80 0x7f2b50157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 --2- 10.20.193.18:0/2097059837 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b500c8b00 0x7f2b501f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 >> 10.20.193.18:0/2097059837 conn(0x7f2b500c9f40 msgr2=0x7f2b500701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 shutdown_connections 2026-02-24T18:35:12.895 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 -- 10.20.193.18:0/2097059837 wait complete. 2026-02-24T18:35:12.896 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:12.891+0000 7f2b56843640 1 librados: shutdown 2026-02-24T18:35:12.902 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:35:19.111 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:57.056 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.113 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:57.059 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.120 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:01:57.066 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.142 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:57.087 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.145 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:57.090 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.145 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:57.053 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.146 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:57.054 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.151 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:01:57.097 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:19.153 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:01:57.062 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:20.516 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:20.582 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:20.582 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 10 monclient: build_initial_monmap 2026-02-24T18:35:20.582 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 10 monclient: monmap: 2026-02-24T18:35:20.582 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:20.582 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:35:20.582139+0000 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:35:20.582139+0000 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 1 Processor -- start 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 1 -- start start 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.581+0000 7f1c78aa9640 10 monclient: init 2026-02-24T18:35:20.583 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient: _add_conns ranks=[2,5,3,0,4,1] 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c74155c00 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1c74155fe0 0x7f1c7415f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-c con 0x7f1c74155fe0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c740c7e40 0x7f1c7415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-a con 0x7f1c740c7e40 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c740c82c0 con 0x7f1c74155c00 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1c740c85a0 con 0x7f1c740c7e40 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1c740c8880 con 0x7f1c74155fe0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c78aa9640 10 monclient(hunting): authenticate will time out at 1464.263300s 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.582+0000 7f1c72d76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c740c7e40 0x7f1c7415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c740c7e40 0x7f1c7415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46654/0 (socket says 10.20.193.18:46654) 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 1 -- 10.20.193.18:0/2314418102 learned_addr learned my addr 10.20.193.18:0/2314418102 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): get_auth_request con 0x7f1c740c7e40 auth_method 0 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:20.584 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:20.585 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:20.585 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:20.585 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:20.585 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.583+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 10 monclient(hunting): handle_auth_done global_id 4323 payload 274 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 1 -- 10.20.193.18:0/2314418102 >> v2:10.20.193.110:3300/0 conn(0x7f1c74155fe0 msgr2=0x7f1c7415f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 1 --2- 10.20.193.18:0/2314418102 >> v2:10.20.193.110:3300/0 conn(0x7f1c74155fe0 0x7f1c7415f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 1 -- 10.20.193.18:0/2314418102 >> v1:10.20.193.110:6789/0 conn(0x7f1c74155c00 legacy=0x7f1c7415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 1 monclient: found mon.noname-a 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.584+0000 7f1c72d76640 1 -- 10.20.193.18:0/2314418102 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c740c8b60 con 0x7f1c740c7e40 2026-02-24T18:35:20.586 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.585+0000 7f1c72d76640 1 --2- 10.20.193.18:0/2314418102 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c740c7e40 0x7f1c7415f830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c74156590 tx=0x7f1c6402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8069961c2658075b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 1 -- 10.20.193.18:0/2314418102 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402fc50 con 0x7f1c740c7e40 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: dump: 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:20.596 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: _finish_auth 0 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 1 -- 10.20.193.18:0/2314418102 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6402fdf0 con 0x7f1c740c7e40 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 5 monclient: authenticate success, global_id 4323 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:20.597 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 1 -- 10.20.193.18:0/2314418102 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64037610 con 0x7f1c740c7e40 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c71573640 10 monclient: dump: 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:20.598 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 4 set_mon_vals no callback set 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 10 monclient: shutdown 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 -- 10.20.193.18:0/2314418102 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c740c7e40 msgr2=0x7f1c7415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/2314418102 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c740c7e40 0x7f1c7415f830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c74156590 tx=0x7f1c6402f230 comp rx=0 tx=0).stop 2026-02-24T18:35:20.599 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 -- 10.20.193.18:0/2314418102 shutdown_connections 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/2314418102 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c740c7e40 0x7f1c7415f830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/2314418102 >> v2:10.20.193.110:3300/0 conn(0x7f1c74155fe0 0x7f1c7415f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 -- 10.20.193.18:0/2314418102 >> 10.20.193.18:0/2314418102 conn(0x7f1c740c9f40 msgr2=0x7f1c740ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 -- 10.20.193.18:0/2314418102 shutdown_connections 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 1 -- 10.20.193.18:0/2314418102 wait complete. 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 10 monclient: build_initial_monmap 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.591+0000 7f1c78aa9640 10 monclient: monmap: 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:20.600 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.601 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 librados: starting msgr at 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 librados: starting objecter 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 Processor -- start 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 -- start start 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 librados: setting wanted keys 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 librados: calling monclient init 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient: init 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-b con 0x7f1c740c7e40 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c74155c00 0x7f1c741f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.602 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-c con 0x7f1c74155c00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c74155fe0 0x7f1c741f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): picked mon.noname-a con 0x7f1c74155fe0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c740c9640 con 0x7f1c74155fe0 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74073310 con 0x7f1c740c7e40 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741f78f0 con 0x7f1c74155c00 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.592+0000 7f1c78aa9640 10 monclient(hunting): authenticate will time out at 1464.273653s 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:20.603 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c74155fe0 0x7f1c741f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:45436/0 (socket says 10.20.193.18:45436) 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 1 -- 10.20.193.18:0/589183992 learned_addr learned my addr 10.20.193.18:0/589183992 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): get_auth_request con 0x7f1c740c7e40 auth_method 0 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): get_auth_request con 0x7f1c74155fe0 auth_method 0 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:20.604 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:20.605 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:20.605 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:20.605 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:20.605 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.593+0000 7f1c72d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 10 monclient(hunting): handle_auth_done global_id 4321 payload 824 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 -- 10.20.193.18:0/589183992 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c74155c00 msgr2=0x7f1c741f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c74155c00 0x7f1c741f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 -- 10.20.193.18:0/589183992 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c74155fe0 msgr2=0x7f1c741f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c74155fe0 0x7f1c741f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 monclient: found mon.noname-b 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:20.608 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.595+0000 7f1c72575640 1 -- 10.20.193.18:0/589183992 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c741f7bd0 con 0x7f1c740c7e40 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.596+0000 7f1c72575640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b570 tx=0x7f1c6800ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.596+0000 7f1c5affd640 10 client.?.objecter ms_handle_connect 0x7f1c740c7e40 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.596+0000 7f1c5affd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68013020 con 0x7f1c740c7e40 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: dump: 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.609 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: _finish_auth 0 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 5 monclient: authenticate success, global_id 4321 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 10 monclient: _renew_subs 2026-02-24T18:35:20.610 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741f7eb0 con 0x7f1c740c7e40 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 10 client.4321.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 10 monclient: _renew_subs 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741f8470 con 0x7f1c740c7e40 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c78aa9640 1 librados: init done 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c68004480 con 0x7f1c740c7e40 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68002980 con 0x7f1c740c7e40 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:20.611 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c5affd640 10 monclient: dump: 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.597+0000 7f1c70d72640 4 set_mon_vals no callback set 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.598+0000 7f1c78aa9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:20.612 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.598+0000 7f1c78aa9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:20.613 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.598+0000 7f1c78aa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:20.613 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.598+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7415d1f0 con 0x7f1c740c7e40 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.610+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f1c68002b20 con 0x7f1c740c7e40 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.610+0000 7f1c5affd640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 0x7f1c4007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.611+0000 7f1c71d74640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 0x7f1c4007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.611+0000 7f1c71d74640 10 monclient: get_auth_request con 0x7f1c40078d70 auth_method 0 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.611+0000 7f1c71d74640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 0x7f1c4007b230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c7414e7b0 tx=0x7f1c5c006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:20.614 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.611+0000 7f1c5affd640 10 client.4321.objecter ms_handle_connect 0x7f1c40078d70 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.715+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f1c680978b0 con 0x7f1c740c7e40 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.715+0000 7f1c5affd640 10 client.4321.objecter ms_dispatch2 0x7f1c74000b90 osd_map(18..18 src has 1..18) 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.715+0000 7f1c5affd640 3 client.4321.objecter handle_osd_map got epochs [18,18] > 0 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.715+0000 7f1c5affd640 3 client.4321.objecter handle_osd_map decoding full epoch 18 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.715+0000 7f1c5affd640 20 client.4321.objecter dump_active .. 0 homeless 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.716+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c6801d020 con 0x7f1c740c7e40 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.716+0000 7f1c5affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:20.718 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.716+0000 7f1c5affd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:20.807 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.805+0000 7f1c78aa9640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:20.807 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.805+0000 7f1c78aa9640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:20.807 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.805+0000 7f1c78aa9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:20.807 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.805+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c74159720 con 0x7f1c740c7e40 2026-02-24T18:35:20.817 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.816+0000 7f1c5affd640 1 -- 10.20.193.18:0/589183992 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f1c680626a0 con 0x7f1c740c7e40 2026-02-24T18:35:20.817 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.816+0000 7f1c5affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:20.818 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.816+0000 7f1c5affd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:20.818 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:35:20.818 INFO:teuthology.misc.health.trial018.stdout:{"epoch":18,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:33:51.286558+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:20.820 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 10 librados: watch_flush enter 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 10 librados: watch_flush exit 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 20 client.4321.objecter shutdown clearing up homeless session... 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 10 client.4321.objecter successfully canceled tick 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 msgr2=0x7f1c4007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 0x7f1c4007b230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c7414e7b0 tx=0x7f1c5c006cf0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 10 monclient: shutdown 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 msgr2=0x7f1c740c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b570 tx=0x7f1c6800ba40 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 shutdown_connections 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c40078d70 0x7f1c4007b230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c74155fe0 0x7f1c741f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c74155c00 0x7f1c741f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 --2- 10.20.193.18:0/589183992 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c740c7e40 0x7f1c740c6580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 >> 10.20.193.18:0/589183992 conn(0x7f1c740c9f40 msgr2=0x7f1c7415dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 shutdown_connections 2026-02-24T18:35:20.821 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 -- 10.20.193.18:0/589183992 wait complete. 2026-02-24T18:35:20.822 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:20.818+0000 7f1c78aa9640 1 librados: shutdown 2026-02-24T18:35:20.828 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-24T18:35:21.874 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:35:21.871+0000 77b8940 -1 osd.2 0 log_to_monitors true 2026-02-24T18:35:21.996 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:35:21.993+0000 77b8940 -1 osd.6 0 log_to_monitors true 2026-02-24T18:35:22.076 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:35:22.074+0000 77b8940 -1 osd.10 0 log_to_monitors true 2026-02-24T18:35:22.559 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:35:22.557+0000 77b8940 -1 osd.14 0 log_to_monitors true 2026-02-24T18:35:22.709 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:35:22.705+0000 77b8940 -1 osd.11 0 log_to_monitors true 2026-02-24T18:35:23.085 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:35:23.081+0000 77b8940 -1 osd.1 0 log_to_monitors true 2026-02-24T18:35:23.133 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:35:23.130+0000 77b8940 -1 osd.4 0 log_to_monitors true 2026-02-24T18:35:23.264 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:35:23.259+0000 77b8940 -1 osd.15 0 log_to_monitors true 2026-02-24T18:35:23.331 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:35:23.325+0000 77b8940 -1 osd.8 0 log_to_monitors true 2026-02-24T18:35:23.343 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:35:23.336+0000 77b8940 -1 osd.5 0 log_to_monitors true 2026-02-24T18:35:23.349 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:35:23.345+0000 77b8940 -1 osd.3 0 log_to_monitors true 2026-02-24T18:35:23.464 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:35:23.460+0000 77b8940 -1 osd.9 0 log_to_monitors true 2026-02-24T18:35:23.600 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:35:23.593+0000 77b8940 -1 osd.0 0 log_to_monitors true 2026-02-24T18:35:23.618 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:35:23.615+0000 77b8940 -1 osd.12 0 log_to_monitors true 2026-02-24T18:35:23.716 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:35:23.712+0000 77b8940 -1 osd.7 0 log_to_monitors true 2026-02-24T18:35:23.862 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:35:23.860+0000 d2b0640 -1 osd.10 0 waiting for initial osdmap 2026-02-24T18:35:23.888 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:35:23.886+0000 dab1640 -1 osd.2 0 waiting for initial osdmap 2026-02-24T18:35:23.899 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:35:23.897+0000 dab1640 -1 osd.6 0 waiting for initial osdmap 2026-02-24T18:35:23.992 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:35:23.988+0000 77b8940 -1 osd.13 0 log_to_monitors true 2026-02-24T18:35:24.098 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:35:24.091+0000 1607e640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:24.103 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:35:24.096+0000 11c76640 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:24.113 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:35:24.107+0000 11c76640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:24.899 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:35:24.896+0000 dab1640 -1 osd.14 0 waiting for initial osdmap 2026-02-24T18:35:24.906 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:35:24.901+0000 dab1640 -1 osd.11 0 waiting for initial osdmap 2026-02-24T18:35:24.981 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:35:24.977+0000 d2b0640 -1 osd.1 0 waiting for initial osdmap 2026-02-24T18:35:24.997 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:35:24.994+0000 d2b0640 -1 osd.4 0 waiting for initial osdmap 2026-02-24T18:35:25.137 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:35:25.130+0000 11c76640 -1 osd.14 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:25.148 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:35:25.139+0000 15c7e640 -1 osd.11 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:25.179 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:35:25.170+0000 11c76640 -1 osd.1 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:25.193 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:35:25.186+0000 11c76640 -1 osd.4 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:25.898 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:35:25.894+0000 d2b0640 -1 osd.0 0 waiting for initial osdmap 2026-02-24T18:35:25.912 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:35:25.908+0000 dab1640 -1 osd.12 0 waiting for initial osdmap 2026-02-24T18:35:25.917 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:35:25.913+0000 d2b0640 -1 osd.3 0 waiting for initial osdmap 2026-02-24T18:35:25.929 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:35:25.925+0000 d2b0640 -1 osd.5 0 waiting for initial osdmap 2026-02-24T18:35:25.938 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:35:25.934+0000 d2b0640 -1 osd.9 0 waiting for initial osdmap 2026-02-24T18:35:25.940 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:35:25.936+0000 dab1640 -1 osd.13 0 waiting for initial osdmap 2026-02-24T18:35:25.943 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:35:25.940+0000 d2b0640 -1 osd.8 0 waiting for initial osdmap 2026-02-24T18:35:25.958 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:35:25.953+0000 dab1640 -1 osd.15 0 waiting for initial osdmap 2026-02-24T18:35:25.968 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:35:25.963+0000 dab1640 -1 osd.7 0 waiting for initial osdmap 2026-02-24T18:35:26.147 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:35:26.139+0000 11c76640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.151 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:35:26.141+0000 15c7e640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.162 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:35:26.155+0000 15c7e640 -1 osd.12 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.163 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:35:26.154+0000 15c7e640 -1 osd.9 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.169 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:35:26.160+0000 15c7e640 -1 osd.5 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.222 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:35:26.215+0000 11c76640 -1 osd.8 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.225 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:35:26.215+0000 15c7e640 -1 osd.15 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.225 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:35:26.216+0000 11c76640 -1 osd.13 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:26.250 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:35:26.241+0000 15c7e640 -1 osd.7 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:35:28.441 DEBUG:teuthology.orchestra.run.trial018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.501+0000 7f6e1bd08640 10 monclient: get_monmap_and_config 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.501+0000 7f6e1bd08640 10 monclient: build_initial_monmap 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.501+0000 7f6e1bd08640 10 monclient: monmap: 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:35:28.502686+0000 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:35:28.502686+0000 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:28.503 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 Processor -- start 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 -- start start 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient: init 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient: _add_conns ranks=[4,5,2,0,3,1] 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6e141539a0 0x7f6e140c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-b con 0x7f6e141539a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 0x7f6e14074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.504 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-c con 0x7f6e14153d80 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e140c8b00 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e140c5710 con 0x7f6e140c8b00 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6e140c5cd0 con 0x7f6e141539a0 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6e140c59f0 con 0x7f6e14153d80 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.502+0000 7f6e1bd08640 10 monclient(hunting): authenticate will time out at 1472.183814s 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 0x7f6e14074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.505 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e19a7d640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6e141539a0 0x7f6e140c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 0x7f6e14074960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57514/0 (socket says 10.20.193.18:57514) 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 1 -- 10.20.193.18:0/4270456704 learned_addr learned my addr 10.20.193.18:0/4270456704 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 10 monclient(hunting): get_auth_request con 0x7f6e14153d80 auth_method 0 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e1927c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e19a7d640 10 monclient(hunting): get_auth_request con 0x7f6e141539a0 auth_method 0 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e19a7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e19a7d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.507 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.503+0000 7f6e19a7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.504+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_done global_id 4366 payload 274 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 1 -- 10.20.193.18:0/4270456704 >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 msgr2=0x7f6e14074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 1 --2- 10.20.193.18:0/4270456704 >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 0x7f6e14074960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 1 -- 10.20.193.18:0/4270456704 >> v1:10.20.193.110:6789/0 conn(0x7f6e140c8b00 legacy=0x7f6e140c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 10 monclient: _finish_hunting 0 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 1 monclient: found mon.noname-b 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:28.508 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.509 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.505+0000 7f6e19a7d640 1 -- 10.20.193.18:0/4270456704 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e140c8020 con 0x7f6e141539a0 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.506+0000 7f6e19a7d640 1 --2- 10.20.193.18:0/4270456704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e141539a0 0x7f6e140c7270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e1406f660 tx=0x7f6e0402ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f673cbbffebff46f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 1 -- 10.20.193.18:0/4270456704 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0403c070 con 0x7f6e141539a0 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: dump: 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.514 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: _finish_auth 0 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: _check_auth_tickets 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 1 -- 10.20.193.18:0/4270456704 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0402fa60 con 0x7f6e141539a0 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 5 monclient: authenticate success, global_id 4366 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:28.515 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 1 -- 10.20.193.18:0/4270456704 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402fe30 con 0x7f6e141539a0 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e18a7b640 10 monclient: dump: 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.516 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 4 set_mon_vals no callback set 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 10 monclient: shutdown 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 -- 10.20.193.18:0/4270456704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e141539a0 msgr2=0x7f6e140c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/4270456704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e141539a0 0x7f6e140c7270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e1406f660 tx=0x7f6e0402ed50 comp rx=0 tx=0).stop 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 -- 10.20.193.18:0/4270456704 shutdown_connections 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/4270456704 >> v2:10.20.193.110:3300/0 conn(0x7f6e14153d80 0x7f6e14074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/4270456704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e141539a0 0x7f6e140c7270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 -- 10.20.193.18:0/4270456704 >> 10.20.193.18:0/4270456704 conn(0x7f6e140c9f40 msgr2=0x7f6e140ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:28.517 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 -- 10.20.193.18:0/4270456704 shutdown_connections 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.508+0000 7f6e1bd08640 1 -- 10.20.193.18:0/4270456704 wait complete. 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 10 monclient: build_initial_monmap 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 10 monclient: monmap: 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:epoch 0 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:created 0.000000 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:28.518 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 librados: starting msgr at 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 librados: starting objecter 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 Processor -- start 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 -- start start 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 librados: setting wanted keys 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 1 librados: calling monclient init 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.509+0000 7f6e1bd08640 10 monclient: init 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-b con 0x7f6e140c8b00 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e141539a0 0x7f6e141f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.519 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-c con 0x7f6e141539a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e14153d80 0x7f6e141f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): picked mon.noname-a con 0x7f6e14153d80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140c77b0 con 0x7f6e14153d80 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140c85e0 con 0x7f6e140c8b00 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140c52d0 con 0x7f6e141539a0 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1bd08640 10 monclient(hunting): authenticate will time out at 1472.191064s 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.520 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e141539a0 0x7f6e141f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1a27e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e14153d80 0x7f6e141f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:45514/0 (socket says 10.20.193.18:45514) 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 1 -- 10.20.193.18:0/932439755 learned_addr learned my addr 10.20.193.18:0/932439755 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): get_auth_request con 0x7f6e140c8b00 auth_method 0 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): get_auth_request con 0x7f6e141539a0 auth_method 0 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1a27e640 10 monclient(hunting): get_auth_request con 0x7f6e14153d80 auth_method 0 2026-02-24T18:35:28.521 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1a27e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1a27e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1a27e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.510+0000 7f6e1927c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.511+0000 7f6e1a27e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.511+0000 7f6e1a27e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.511+0000 7f6e1a27e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 10 monclient(hunting): handle_auth_done global_id 4369 payload 824 2026-02-24T18:35:28.522 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 -- 10.20.193.18:0/932439755 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e141539a0 msgr2=0x7f6e141f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e141539a0 0x7f6e141f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 -- 10.20.193.18:0/932439755 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e14153d80 msgr2=0x7f6e141f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e14153d80 0x7f6e141f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 10 monclient: _finish_hunting 0 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 monclient: found mon.noname-b 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.523 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.512+0000 7f6e19a7d640 1 -- 10.20.193.18:0/932439755 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141f2820 con 0x7f6e140c8b00 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.513+0000 7f6e19a7d640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009130 tx=0x7f6e040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.513+0000 7f6e027fc640 10 client.?.objecter ms_handle_connect 0x7f6e140c8b00 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.513+0000 7f6e027fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0403c040 con 0x7f6e140c8b00 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: dump: 2026-02-24T18:35:28.525 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:28.526 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e04042450 con 0x7f6e140c8b00 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04042820 con 0x7f6e140c8b00 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e027fc640 10 monclient: dump: 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:epoch 1 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.527 INFO:teuthology.misc.health.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr: 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e03fff640 4 set_mon_vals no callback set 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 5 monclient: authenticate success, global_id 4369 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 10 monclient: _renew_subs 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e14151f50 con 0x7f6e140c8b00 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 10 client.4369.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 10 monclient: _renew_subs 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e14152510 con 0x7f6e140c8b00 2026-02-24T18:35:28.528 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.514+0000 7f6e1bd08640 1 librados: init done 2026-02-24T18:35:28.529 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.515+0000 7f6e1bd08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.529 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.515+0000 7f6e1bd08640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.529 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.515+0000 7f6e1bd08640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.529 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.515+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e140c7270 con 0x7f6e140c8b00 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.526+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f6e040429c0 con 0x7f6e140c8b00 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.526+0000 7f6e027fc640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 0x7f6de407b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.526+0000 7f6e1927c640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 0x7f6de407b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.526+0000 7f6e1927c640 10 monclient: get_auth_request con 0x7f6de4078da0 auth_method 0 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.527+0000 7f6e1927c640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 0x7f6de407b260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e14146160 tx=0x7f6e08009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.527+0000 7f6e027fc640 10 client.4369.objecter ms_handle_connect 0x7f6de4078da0 2026-02-24T18:35:28.530 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.528+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 8563+0+0 (secure 0 0 0) 0x7f6e040bcd10 con 0x7f6e140c8b00 2026-02-24T18:35:28.531 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.528+0000 7f6e027fc640 10 client.4369.objecter ms_dispatch2 0x7f6e14000b90 osd_map(24..24 src has 1..24) 2026-02-24T18:35:28.531 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.528+0000 7f6e027fc640 3 client.4369.objecter handle_osd_map got epochs [24,24] > 0 2026-02-24T18:35:28.531 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.528+0000 7f6e027fc640 3 client.4369.objecter handle_osd_map decoding full epoch 24 2026-02-24T18:35:28.531 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.528+0000 7f6e027fc640 20 client.4369.objecter dump_active .. 0 homeless 2026-02-24T18:35:28.628 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.626+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e040875c0 con 0x7f6e140c8b00 2026-02-24T18:35:28.628 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.626+0000 7f6e027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.628 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.626+0000 7f6e027fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:28.711 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.709+0000 7f6e1bd08640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:28.711 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.709+0000 7f6e1bd08640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:28.711 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.709+0000 7f6e1bd08640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:28.711 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.709+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e140cb340 con 0x7f6e140c8b00 2026-02-24T18:35:28.757 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.756+0000 7f6e027fc640 1 -- 10.20.193.18:0/932439755 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+22721 (secure 0 0 0) 0x7f6e0408b8c0 con 0x7f6e140c8b00 2026-02-24T18:35:28.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.756+0000 7f6e027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:28.758 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.756+0000 7f6e027fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:28.758 INFO:teuthology.misc.health.trial018.stdout: 2026-02-24T18:35:28.758 INFO:teuthology.misc.health.trial018.stdout:{"epoch":24,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:35:28.068886+0000","last_up_change":"2026-02-24T18:35:26.937667+0000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:35:26.061331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6817","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6819","nonce":1076077393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6821","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6823","nonce":1076077393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6829","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6831","nonce":1076077393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6825","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6827","nonce":1076077393}]},"public_addr":"10.20.193.18:6819/1076077393","cluster_addr":"10.20.193.18:6823/1076077393","heartbeat_back_addr":"10.20.193.18:6831/1076077393","heartbeat_front_addr":"10.20.193.18:6827/1076077393","state":["exists","up"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6800","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6801","nonce":2795974188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6802","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6803","nonce":2795974188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6806","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6807","nonce":2795974188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6804","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6805","nonce":2795974188}]},"public_addr":"10.20.193.32:6801/2795974188","cluster_addr":"10.20.193.32:6803/2795974188","heartbeat_back_addr":"10.20.193.32:6807/2795974188","heartbeat_front_addr":"10.20.193.32:6805/2795974188","state":["exists","up"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6825","nonce":3650974883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6826","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6827","nonce":3650974883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6830","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6831","nonce":3650974883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6829","nonce":3650974883}]},"public_addr":"10.20.193.110:6825/3650974883","cluster_addr":"10.20.193.110:6827/3650974883","heartbeat_back_addr":"10.20.193.110:6831/3650974883","heartbeat_front_addr":"10.20.193.110:6829/3650974883","state":["exists","up"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6826","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6827","nonce":230320724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6828","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6829","nonce":230320724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6832","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6833","nonce":230320724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6830","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6831","nonce":230320724}]},"public_addr":"10.20.193.112:6827/230320724","cluster_addr":"10.20.193.112:6829/230320724","heartbeat_back_addr":"10.20.193.112:6833/230320724","heartbeat_front_addr":"10.20.193.112:6831/230320724","state":["exists","up"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6800","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6801","nonce":3481454300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6802","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6803","nonce":3481454300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6806","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6807","nonce":3481454300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6804","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6805","nonce":3481454300}]},"public_addr":"10.20.193.18:6801/3481454300","cluster_addr":"10.20.193.18:6803/3481454300","heartbeat_back_addr":"10.20.193.18:6807/3481454300","heartbeat_front_addr":"10.20.193.18:6805/3481454300","state":["exists","up"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6808","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6809","nonce":2128834550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6810","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6811","nonce":2128834550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6814","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6815","nonce":2128834550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6812","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6813","nonce":2128834550}]},"public_addr":"10.20.193.32:6809/2128834550","cluster_addr":"10.20.193.32:6811/2128834550","heartbeat_back_addr":"10.20.193.32:6815/2128834550","heartbeat_front_addr":"10.20.193.32:6813/2128834550","state":["exists","up"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6801","nonce":3466222224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6803","nonce":3466222224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6807","nonce":3466222224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6805","nonce":3466222224}]},"public_addr":"10.20.193.110:6801/3466222224","cluster_addr":"10.20.193.110:6803/3466222224","heartbeat_back_addr":"10.20.193.110:6807/3466222224","heartbeat_front_addr":"10.20.193.110:6805/3466222224","state":["exists","up"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6810","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6811","nonce":92089351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6814","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6815","nonce":92089351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6821","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6823","nonce":92089351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6817","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6819","nonce":92089351}]},"public_addr":"10.20.193.112:6811/92089351","cluster_addr":"10.20.193.112:6815/92089351","heartbeat_back_addr":"10.20.193.112:6823/92089351","heartbeat_front_addr":"10.20.193.112:6819/92089351","state":["exists","up"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6808","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6809","nonce":1977979333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6810","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6811","nonce":1977979333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6814","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6815","nonce":1977979333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6812","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6813","nonce":1977979333}]},"public_addr":"10.20.193.18:6809/1977979333","cluster_addr":"10.20.193.18:6811/1977979333","heartbeat_back_addr":"10.20.193.18:6815/1977979333","heartbeat_front_addr":"10.20.193.18:6813/1977979333","state":["exists","up"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6816","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6817","nonce":1806399530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6818","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6819","nonce":1806399530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6822","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6823","nonce":1806399530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6820","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6821","nonce":1806399530}]},"public_addr":"10.20.193.32:6817/1806399530","cluster_addr":"10.20.193.32:6819/1806399530","heartbeat_back_addr":"10.20.193.32:6823/1806399530","heartbeat_front_addr":"10.20.193.32:6821/1806399530","state":["exists","up"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6809","nonce":4149932150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6811","nonce":4149932150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6815","nonce":4149932150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6813","nonce":4149932150}]},"public_addr":"10.20.193.110:6809/4149932150","cluster_addr":"10.20.193.110:6811/4149932150","heartbeat_back_addr":"10.20.193.110:6815/4149932150","heartbeat_front_addr":"10.20.193.110:6813/4149932150","state":["exists","up"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6812","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6813","nonce":2071089960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6816","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6818","nonce":2071089960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6824","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6825","nonce":2071089960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6820","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6822","nonce":2071089960}]},"public_addr":"10.20.193.112:6813/2071089960","cluster_addr":"10.20.193.112:6818/2071089960","heartbeat_back_addr":"10.20.193.112:6825/2071089960","heartbeat_front_addr":"10.20.193.112:6822/2071089960","state":["exists","up"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6816","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6818","nonce":392025864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6820","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6822","nonce":392025864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6828","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6830","nonce":392025864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6824","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6826","nonce":392025864}]},"public_addr":"10.20.193.18:6818/392025864","cluster_addr":"10.20.193.18:6822/392025864","heartbeat_back_addr":"10.20.193.18:6830/392025864","heartbeat_front_addr":"10.20.193.18:6826/392025864","state":["exists","up"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6824","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6825","nonce":316721302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6826","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6827","nonce":316721302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6830","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6831","nonce":316721302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6828","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6829","nonce":316721302}]},"public_addr":"10.20.193.32:6825/316721302","cluster_addr":"10.20.193.32:6827/316721302","heartbeat_back_addr":"10.20.193.32:6831/316721302","heartbeat_front_addr":"10.20.193.32:6829/316721302","state":["exists","up"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6817","nonce":3577843273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6818","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6819","nonce":3577843273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6822","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6823","nonce":3577843273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6821","nonce":3577843273}]},"public_addr":"10.20.193.110:6817/3577843273","cluster_addr":"10.20.193.110:6819/3577843273","heartbeat_back_addr":"10.20.193.110:6823/3577843273","heartbeat_front_addr":"10.20.193.110:6821/3577843273","state":["exists","up"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6802","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6803","nonce":4220209204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6804","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6805","nonce":4220209204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6808","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6809","nonce":4220209204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6806","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6807","nonce":4220209204}]},"public_addr":"10.20.193.112:6803/4220209204","cluster_addr":"10.20.193.112:6805/4220209204","heartbeat_back_addr":"10.20.193.112:6809/4220209204","heartbeat_front_addr":"10.20.193.112:6807/4220209204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.519970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.248082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.777136+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.483378+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.749510+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.414866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.123257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.766447+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.874254+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.504376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.120504+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.378335+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.904134+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.610284+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 10 librados: watch_flush enter 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 10 librados: watch_flush exit 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 20 client.4369.objecter shutdown clearing up homeless session... 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 10 client.4369.objecter successfully canceled tick 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 msgr2=0x7f6de407b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 0x7f6de407b260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e14146160 tx=0x7f6e08009340 comp rx=0 tx=0).stop 2026-02-24T18:35:28.760 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 10 monclient: shutdown 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 msgr2=0x7f6e141f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009130 tx=0x7f6e040047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 shutdown_connections 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6de4078da0 0x7f6de407b260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e14153d80 0x7f6e141f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e141539a0 0x7f6e141f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 --2- 10.20.193.18:0/932439755 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e140c8b00 0x7f6e141f3fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 >> 10.20.193.18:0/932439755 conn(0x7f6e140c9f40 msgr2=0x7f6e140c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 shutdown_connections 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 -- 10.20.193.18:0/932439755 wait complete. 2026-02-24T18:35:28.761 INFO:teuthology.misc.health.trial018.stderr:2026-02-24T18:35:28.757+0000 7f6e1bd08640 1 librados: shutdown 2026-02-24T18:35:28.767 DEBUG:teuthology.misc:16 of 16 OSDs are up 2026-02-24T18:35:28.767 INFO:tasks.ceph:Creating RBD pool 2026-02-24T18:35:28.767 DEBUG:teuthology.orchestra.run.trial018:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-24T18:35:28.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 10 monclient: get_monmap_and_config 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 10 monclient: build_initial_monmap 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 10 monclient: monmap: 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:28.834030+0000 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:28.834030+0000 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:28.835 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:28.836 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:28.836 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:28.836 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:28.836 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:28.836 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 1 Processor -- start 2026-02-24T18:35:28.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 1 -- start start 2026-02-24T18:35:28.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.833+0000 7fae36658640 10 monclient: init 2026-02-24T18:35:28.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient: _add_conns ranks=[5,4,3,2,0,1] 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fae301527d0 0x7fae3006da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): picked mon.noname-c con 0x7fae301527d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fae30152bb0 0x7fae300c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): picked mon.noname-b con 0x7fae30152bb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fae300c5fc0 0x7fae300c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): picked mon.noname-a con 0x7fae300c5fc0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fae3006f1f0 con 0x7fae300c5fc0 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fae300c6470 con 0x7fae30152bb0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fae3006e300 con 0x7fae301527d0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae36658640 10 monclient(hunting): authenticate will time out at 1472.515246s 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae34bce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fae300c5fc0 0x7fae300c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fae301527d0 0x7fae3006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2f7fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fae30152bb0 0x7fae300c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fae301527d0 0x7fae3006da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57540/0 (socket says 10.20.193.18:57540) 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 1 -- 10.20.193.18:0/1172829506 learned_addr learned my addr 10.20.193.18:0/1172829506 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 10 monclient(hunting): get_auth_request con 0x7fae301527d0 auth_method 0 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae34bce640 10 monclient(hunting): get_auth_request con 0x7fae300c5fc0 auth_method 0 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae34bce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae34bce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.834+0000 7fae34bce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): get_auth_request con 0x7fae30152bb0 auth_method 0 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.835+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 10 monclient(hunting): handle_auth_done global_id 4370 payload 274 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 -- 10.20.193.18:0/1172829506 >> v2:10.20.193.32:3300/0 conn(0x7fae30152bb0 msgr2=0x7fae300c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 --2- 10.20.193.18:0/1172829506 >> v2:10.20.193.32:3300/0 conn(0x7fae30152bb0 0x7fae300c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 -- 10.20.193.18:0/1172829506 >> v2:10.20.193.18:3300/0 conn(0x7fae300c5fc0 msgr2=0x7fae300c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 --2- 10.20.193.18:0/1172829506 >> v2:10.20.193.18:3300/0 conn(0x7fae300c5fc0 0x7fae300c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 monclient: found mon.noname-c 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:28.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:28.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.836+0000 7fae2ffff640 1 -- 10.20.193.18:0/1172829506 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3006e5e0 con 0x7fae301527d0 2026-02-24T18:35:28.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.837+0000 7fae2ffff640 1 --2- 10.20.193.18:0/1172829506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae3006da60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae300750e0 tx=0x7fae1c02ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b08024c4996379 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 1 -- 10.20.193.18:0/1172829506 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c03c070 con 0x7fae301527d0 2026-02-24T18:35:28.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:28.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: dump: 2026-02-24T18:35:28.850 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: _finish_auth 0 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:28.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 1 -- 10.20.193.18:0/1172829506 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c02fb50 con 0x7fae301527d0 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 5 monclient: authenticate success, global_id 4370 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 1 -- 10.20.193.18:0/1172829506 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c037440 con 0x7fae301527d0 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 4 set_mon_vals no callback set 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae2effd640 10 monclient: dump: 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:28.852 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 10 monclient: shutdown 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 1 -- 10.20.193.18:0/1172829506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 msgr2=0x7fae3006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.839+0000 7fae36658640 1 --2- 10.20.193.18:0/1172829506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae3006da60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae300750e0 tx=0x7fae1c02ee90 comp rx=0 tx=0).stop 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 -- 10.20.193.18:0/1172829506 shutdown_connections 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 --2- 10.20.193.18:0/1172829506 >> v2:10.20.193.18:3300/0 conn(0x7fae300c5fc0 0x7fae300c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 --2- 10.20.193.18:0/1172829506 >> v2:10.20.193.32:3300/0 conn(0x7fae30152bb0 0x7fae300c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 --2- 10.20.193.18:0/1172829506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae3006da60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 -- 10.20.193.18:0/1172829506 >> 10.20.193.18:0/1172829506 conn(0x7fae300cd780 msgr2=0x7fae300cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 -- 10.20.193.18:0/1172829506 shutdown_connections 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 -- 10.20.193.18:0/1172829506 wait complete. 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 10 monclient: build_initial_monmap 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 10 monclient: monmap: 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:28.854 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 librados: starting msgr at 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 librados: starting objecter 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 Processor -- start 2026-02-24T18:35:28.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 -- start start 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 librados: setting wanted keys 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 1 librados: calling monclient init 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.840+0000 7fae36658640 10 monclient: init 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fae300c5fc0 0x7fae30147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): picked mon.noname-b con 0x7fae300c5fc0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae301f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): picked mon.noname-c con 0x7fae301527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 0x7fae301f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): picked mon.noname-a con 0x7fae30152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006ee80 con 0x7fae30152bb0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006eba0 con 0x7fae300c5fc0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae3006dfa0 con 0x7fae301527d0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae36658640 10 monclient(hunting): authenticate will time out at 1472.522296s 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 0x7fae301f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae301f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 0x7fae301f37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46718/0 (socket says 10.20.193.18:46718) 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 1 -- 10.20.193.18:0/488208338 learned_addr learned my addr 10.20.193.18:0/488208338 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 10 monclient(hunting): get_auth_request con 0x7fae30152bb0 auth_method 0 2026-02-24T18:35:28.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae34bce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): get_auth_request con 0x7fae301527d0 auth_method 0 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2ffff640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fae300c5fc0 0x7fae30147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.841+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): get_auth_request con 0x7fae300c5fc0 auth_method 0 2026-02-24T18:35:28.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae34bce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.842+0000 7fae2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 10 monclient(hunting): handle_auth_done global_id 4373 payload 824 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 -- 10.20.193.18:0/488208338 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fae300c5fc0 msgr2=0x7fae30147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fae300c5fc0 0x7fae30147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 -- 10.20.193.18:0/488208338 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 msgr2=0x7fae301f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:28.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 0x7fae301f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.843+0000 7fae2f7fe640 1 -- 10.20.193.18:0/488208338 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae301f5160 con 0x7fae301527d0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.844+0000 7fae2f7fe640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae301f32a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae2000e870 tx=0x7fae2000ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.844+0000 7fae2cff9640 10 client.?.objecter ms_handle_connect 0x7fae301527d0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.844+0000 7fae2cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20017070 con 0x7fae301527d0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: dump: 2026-02-24T18:35:28.860 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:28.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2000c940 con 0x7fae301527d0 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 5 monclient: authenticate success, global_id 4373 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 10 monclient: _renew_subs 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae301f5440 con 0x7fae301527d0 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000cd10 con 0x7fae301527d0 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2e7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae2cff9640 10 monclient: dump: 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:28.862 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 10 client.4373.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 10 monclient: _renew_subs 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301f5a00 con 0x7fae301527d0 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.847+0000 7fae36658640 1 librados: init done 2026-02-24T18:35:28.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.848+0000 7fae36658640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.848+0000 7fae36658640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.848+0000 7fae36658640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:28.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.848+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3006da60 con 0x7fae301527d0 2026-02-24T18:35:28.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.859+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7fae20013470 con 0x7fae301527d0 2026-02-24T18:35:28.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.859+0000 7fae2cff9640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 0x7fae0007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:28.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.859+0000 7fae2ffff640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 0x7fae0007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.859+0000 7fae2ffff640 10 monclient: get_auth_request con 0x7fae00078d70 auth_method 0 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.860+0000 7fae2ffff640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 0x7fae0007b230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae30158940 tx=0x7fae1c039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.860+0000 7fae2cff9640 10 client.4373.objecter ms_handle_connect 0x7fae00078d70 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.861+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 8563+0+0 (secure 0 0 0) 0x7fae2005e260 con 0x7fae301527d0 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.861+0000 7fae2cff9640 10 client.4373.objecter ms_dispatch2 0x7fae30000b90 osd_map(24..24 src has 1..24) 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.861+0000 7fae2cff9640 3 client.4373.objecter handle_osd_map got epochs [24,24] > 0 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.861+0000 7fae2cff9640 3 client.4373.objecter handle_osd_map decoding full epoch 24 2026-02-24T18:35:28.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.861+0000 7fae2cff9640 20 client.4373.objecter dump_active .. 0 homeless 2026-02-24T18:35:28.961 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.959+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae20062220 con 0x7fae301527d0 2026-02-24T18:35:28.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.959+0000 7fae2cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:28.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:28.959+0000 7fae2cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:29.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.045+0000 7fae36658640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-24T18:35:29.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.045+0000 7fae36658640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-24T18:35:29.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.045+0000 7fae36658640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:29.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.045+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fae300ce080 con 0x7fae301527d0 2026-02-24T18:35:29.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.200+0000 7fae2cff9640 1 -- 10.20.193.18:0/488208338 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v25) ==== 109+0+0 (secure 0 0 0) 0x7fae20066520 con 0x7fae301527d0 2026-02-24T18:35:29.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.200+0000 7fae2cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-24T18:35:29.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.200+0000 7fae2cff9640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-24T18:35:29.202 INFO:teuthology.orchestra.run.trial018.stderr:pool 'rbd' created 2026-02-24T18:35:29.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 10 librados: watch_flush enter 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 10 librados: watch_flush exit 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 20 client.4373.objecter shutdown clearing up homeless session... 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 10 client.4373.objecter successfully canceled tick 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 msgr2=0x7fae0007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 0x7fae0007b230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae30158940 tx=0x7fae1c039040 comp rx=0 tx=0).stop 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 10 monclient: shutdown 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 msgr2=0x7fae301f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae301f32a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae2000e870 tx=0x7fae2000ed40 comp rx=0 tx=0).stop 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 shutdown_connections 2026-02-24T18:35:29.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fae00078d70 0x7fae0007b230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fae30152bb0 0x7fae301f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fae301527d0 0x7fae301f32a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 --2- 10.20.193.18:0/488208338 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fae300c5fc0 0x7fae30147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 >> 10.20.193.18:0/488208338 conn(0x7fae300cd780 msgr2=0x7fae300c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 shutdown_connections 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 -- 10.20.193.18:0/488208338 wait complete. 2026-02-24T18:35:29.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.202+0000 7fae36658640 1 librados: shutdown 2026-02-24T18:35:29.213 DEBUG:teuthology.orchestra.run.trial018:> rbd --cluster ceph pool init rbd 2026-02-24T18:35:29.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.244+0000 7f70dcb950c0 10 monclient: get_monmap_and_config 2026-02-24T18:35:29.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.244+0000 7f70dcb950c0 10 monclient: build_initial_monmap 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.244+0000 7f70dcb950c0 10 monclient: monmap: 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:29.245702+0000 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:29.245702+0000 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:29.247 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:29.248 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:29.248 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:29.248 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:29.248 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 Processor -- start 2026-02-24T18:35:29.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 -- start start 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient: init 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient: _reopen_session rank -1 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient: _add_conns ranks=[5,3,4,2,1,0] 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 --2- >> v2:10.20.193.110:3300/0 conn(0x562b42d96670 0x562b42da05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-c con 0x562b42d96670 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 --2- >> v2:10.20.193.18:3300/0 conn(0x562b42d96a50 0x562b42d09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-a con 0x562b42d96a50 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 --2- >> v2:10.20.193.32:3300/0 conn(0x562b42d09b60 0x562b42d0bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-b con 0x562b42d09b60 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x562b42b5b720 con 0x562b42d96a50 2026-02-24T18:35:29.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.245+0000 7f70dcb950c0 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x562b42bafe80 con 0x562b42d09b60 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dcb950c0 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x562b42babdb0 con 0x562b42d96670 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dcb950c0 10 monclient(hunting): _renew_subs 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dcb950c0 10 monclient(hunting): authenticate will time out at 1472.927007s 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x562b42d96a50 0x562b42d09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70db65b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x562b42d96670 0x562b42da05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x562b42d96a50 0x562b42d09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41066/0 (socket says 10.20.193.18:41066) 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 1 -- 10.20.193.18:0/3796770349 learned_addr learned my addr 10.20.193.18:0/3796770349 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:29.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): get_auth_request con 0x562b42d96a50 auth_method 0 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dc8e4640 1 --2- 10.20.193.18:0/3796770349 >> v2:10.20.193.32:3300/0 conn(0x562b42d09b60 0x562b42d0bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70db65b640 10 monclient(hunting): get_auth_request con 0x562b42d96670 auth_method 0 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70db65b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70db65b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70db65b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dc8e4640 10 monclient(hunting): get_auth_request con 0x562b42d09b60 auth_method 0 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dc8e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dc8e4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dc8e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.246+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.247+0000 7f70dae5a640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 -- 10.20.193.18:0/3796770349 >> v2:10.20.193.110:3300/0 conn(0x562b42d96670 msgr2=0x562b42da05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:29.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3796770349 >> v2:10.20.193.110:3300/0 conn(0x562b42d96670 0x562b42da05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 -- 10.20.193.18:0/3796770349 >> v2:10.20.193.32:3300/0 conn(0x562b42d09b60 msgr2=0x562b42d0bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3796770349 >> v2:10.20.193.32:3300/0 conn(0x562b42d09b60 0x562b42d0bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 10 monclient: _finish_hunting 0 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 monclient: found mon.noname-a 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.248+0000 7f70dae5a640 1 -- 10.20.193.18:0/3796770349 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b42b2ed20 con 0x562b42d96a50 2026-02-24T18:35:29.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.249+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3796770349 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d96a50 0x562b42d09620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x562b42bbe4c0 tx=0x7f70c4058670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f955aa3de8e4c1d0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 1 -- 10.20.193.18:0/3796770349 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c406d070 con 0x562b42d96a50 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: dump: 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:29.263 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: _finish_auth 0 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: _check_auth_tickets 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:29.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 1 -- 10.20.193.18:0/3796770349 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c4060d30 con 0x562b42d96a50 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 1 -- 10.20.193.18:0/3796770349 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4068680 con 0x562b42d96a50 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 5 monclient: authenticate success, global_id 4383 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 10 monclient: get_monmap_and_config success 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 4 set_mon_vals no callback set 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70da659640 10 monclient: dump: 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.265 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 10 monclient: shutdown 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3796770349 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d96a50 msgr2=0x562b42d09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.251+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3796770349 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d96a50 0x562b42d09620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x562b42bbe4c0 tx=0x7f70c4058670 comp rx=0 tx=0).stop 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3796770349 shutdown_connections 2026-02-24T18:35:29.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3796770349 >> v2:10.20.193.32:3300/0 conn(0x562b42d09b60 0x562b42d0bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3796770349 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d96a50 0x562b42d09620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3796770349 >> v2:10.20.193.110:3300/0 conn(0x562b42d96670 0x562b42da05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3796770349 >> 10.20.193.18:0/3796770349 conn(0x562b42cc8730 msgr2=0x562b42cc8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3796770349 shutdown_connections 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3796770349 wait complete. 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 10 monclient: build_initial_monmap 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 10 monclient: monmap: 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:29.267 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 librados: starting msgr at 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 librados: starting objecter 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 Processor -- start 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 -- start start 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 librados: setting wanted keys 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 1 librados: calling monclient init 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.252+0000 7f70dcb950c0 10 monclient: init 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient: _reopen_session rank -1 2026-02-24T18:35:29.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-a con 0x562b42d09b60 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x562b42d96670 0x562b42cc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-b con 0x562b42d96670 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x562b42d96a50 0x562b42ccc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): picked mon.noname-c con 0x562b42d96a50 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x562b42d085e0 con 0x562b42d09b60 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x562b42b4ae50 con 0x562b42d96670 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x562b42da0b00 con 0x562b42d96a50 2026-02-24T18:35:29.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): _renew_subs 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dcb950c0 10 monclient(hunting): authenticate will time out at 1472.934298s 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41080/0 (socket says 10.20.193.18:41080) 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 learned_addr learned my addr 10.20.193.18:0/3018835393 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): get_auth_request con 0x562b42d09b60 auth_method 0 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dc8e4640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x562b42d96a50 0x562b42ccc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x562b42d96670 0x562b42cc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dae5a640 10 monclient(hunting): get_auth_request con 0x562b42d96670 auth_method 0 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dae5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dae5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dae5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70db65b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dc8e4640 10 monclient(hunting): get_auth_request con 0x562b42d96a50 auth_method 0 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dc8e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dc8e4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.253+0000 7f70dc8e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dae5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.254+0000 7f70dc8e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 10 monclient(hunting): handle_auth_done global_id 4386 payload 824 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x562b42d96a50 msgr2=0x562b42ccc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x562b42d96a50 0x562b42ccc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x562b42d96670 msgr2=0x562b42cc73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x562b42d96670 0x562b42cc73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 monclient: found mon.noname-a 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:29.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.255+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b42ccb350 con 0x562b42d09b60 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.256+0000 7f70db65b640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70d0008810 tx=0x7f70d0008910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.256+0000 7f70c37fe640 10 client.?.objecter ms_handle_connect 0x562b42d09b60 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.256+0000 7f70c37fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0017020 con 0x562b42d09b60 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: dump: 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.273 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 5 monclient: authenticate success, global_id 4386 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d0013810 con 0x562b42d09b60 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 monclient: _renew_subs 2026-02-24T18:35:29.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562b42ccb5c0 con 0x562b42d09b60 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0013bc0 con 0x562b42d09b60 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70c37fe640 10 monclient: dump: 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:29.275 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70d9e58640 4 set_mon_vals no callback set 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 client.4386.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 monclient: _renew_subs 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b42ccbb40 con 0x562b42d09b60 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 1 librados: init done 2026-02-24T18:35:29.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.258+0000 7f70dcb950c0 10 librados: wait_for_osdmap waiting 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.271+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f70d001c400 con 0x562b42d09b60 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.272+0000 7f70c37fe640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70bc076da0 0x7f70bc079260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.272+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70bc076da0 0x7f70bc079260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.272+0000 7f70dae5a640 10 monclient: get_auth_request con 0x7f70bc076da0 auth_method 0 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.273+0000 7f70dae5a640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70bc076da0 0x7f70bc079260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70c406a000 tx=0x7f70c4064000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.273+0000 7f70c37fe640 10 client.4386.objecter ms_handle_connect 0x7f70bc076da0 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 8905+0+0 (secure 0 0 0) 0x7f70d0096740 con 0x562b42d09b60 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_map(25..25 src has 1..25) 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map got epochs [25,25] > 0 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map decoding full epoch 25 2026-02-24T18:35:29.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70c37fe640 20 client.4386.objecter dump_active .. 0 homeless 2026-02-24T18:35:29.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70dcb950c0 10 librados: wait_for_osdmap done waiting 2026-02-24T18:35:29.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70dcb950c0 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-24T18:35:29.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70dcb950c0 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-24T18:35:29.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70dcb950c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:29.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:29.274+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x562b42cc7dc0 con 0x562b42d09b60 2026-02-24T18:35:30.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.266+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v26) ==== 141+0+0 (secure 0 0 0) 0x7f70d0021030 con 0x562b42d09b60 2026-02-24T18:35:30.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.266+0000 7f70c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-24T18:35:30.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.266+0000 7f70c37fe640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-24T18:35:30.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.266+0000 7f70dcb950c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:30.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.266+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70c4060d30 con 0x562b42d09b60 2026-02-24T18:35:30.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.402+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f70d005ce00 con 0x562b42d09b60 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 8 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70d0013bc0 con 0x562b42d09b60 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70c37fe640 10 monclient: handle_get_version_reply finishing 1 version 26 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70d9e58640 10 client.4386.objecter _get_latest_version latest 26, waiting 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70d9e58640 10 client.4386.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70d9e58640 10 monclient: _renew_subs 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70d9e58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:30.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.415+0000 7f70d9e58640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=26}) -- 0x7f70b4003c90 con 0x562b42d09b60 2026-02-24T18:35:30.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 9 ==== osd_map(26..26 src has 1..26) ==== 682+0+0 (secure 0 0 0) 0x7f70d006ab50 con 0x562b42d09b60 2026-02-24T18:35:30.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_map(26..26 src has 1..26) 2026-02-24T18:35:30.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map got epochs [26,26] > 25 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map decoding incremental epoch 26 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70c37fe640 20 client.4386.objecter dump_active .. 0 homeless 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 10 librados: create oid=rbd_trash nspace= 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 10 client.4386.objecter _op_submit op 0x562b42cc6480 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 20 client.4386.objecter _calc_target epoch 26 base rbd_trash @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 20 client.4386.objecter _calc_target target rbd_trash @2 -> pgid 2.5a749323 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 10 client.4386.objecter _calc_target raw pgid 2.5a749323 -> actual 2.3 acting [4,15] primary 4 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x562b42cc78e0 0x562b42daba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 20 client.4386.objecter _get_session s=0x562b42b4b230 osd=4 3 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 10 client.4386.objecter _op_submit oid rbd_trash '@2' '@2' [create] tid 1 osd.4 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 20 client.4386.objecter get_session s=0x562b42b4b230 osd=4 3 2026-02-24T18:35:30.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 15 client.4386.objecter _session_op_assign 4 1 2026-02-24T18:35:30.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 15 client.4386.objecter _send_op 1 to 2.3 on osd.4 2026-02-24T18:35:30.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) -- 0x562b42dabfc0 con 0x562b42cc78e0 2026-02-24T18:35:30.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 20 client.4386.objecter put_session s=0x562b42b4b230 osd=4 4 2026-02-24T18:35:30.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.425+0000 7f70dcb950c0 5 client.4386.objecter 1 in flight 2026-02-24T18:35:30.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.429+0000 7f70dc8e4640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x562b42cc78e0 0x562b42daba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:30.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.469+0000 7f70dc8e4640 10 monclient: get_auth_request con 0x562b42cc78e0 auth_method 0 2026-02-24T18:35:30.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.531+0000 7f70dc8e4640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x562b42cc78e0 0x562b42daba80 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:30.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.531+0000 7f70c37fe640 10 client.4386.objecter ms_handle_connect 0x562b42cc78e0 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 1 -- 10.20.193.18:0/3018835393 <== osd.4 v2:10.20.193.18:6800/3481454300 1 ==== osd_op_reply(1 rbd_trash [create] v26'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f70cc002a50 con 0x562b42cc78e0 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_op_reply(1 rbd_trash [create] v26'1 uv1 ondisk = 0) 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 10 client.4386.objecter in handle_osd_op_reply 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 7 client.4386.objecter handle_osd_op_reply 1 ondisk uv 1 in 2.3 attempt 0 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 10 client.4386.objecter op 0 rval 0 len 0 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 15 client.4386.objecter handle_osd_op_reply completed tid 1 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 15 client.4386.objecter _finish_op 1 2026-02-24T18:35:30.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 20 client.4386.objecter put_session s=0x562b42b4b230 osd=4 4 2026-02-24T18:35:30.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 15 client.4386.objecter _session_op_remove 4 1 2026-02-24T18:35:30.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dc8e4640 5 client.4386.objecter 0 in flight 2026-02-24T18:35:30.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.892+0000 7f70dcb950c0 10 librados: Objecter returned from create r=0 2026-02-24T18:35:30.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 10 client.4386.objecter _op_submit op 0x562b42e08730 2026-02-24T18:35:30.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 20 client.4386.objecter _calc_target epoch 26 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-24T18:35:30.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 20 client.4386.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-24T18:35:30.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 10 client.4386.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x562b42e092e0 0x562b42e0b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 20 client.4386.objecter _get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 10 client.4386.objecter _op_submit oid rbd_info '@2' '@2' [call rbd.metadata_list in=33b] tid 2 osd.3 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 20 client.4386.objecter get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 15 client.4386.objecter _session_op_assign 3 2 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 15 client.4386.objecter _send_op 2 to 2.2 on osd.3 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) -- 0x562b42e0bcf0 con 0x562b42e092e0 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.893+0000 7f70dcb950c0 5 client.4386.objecter 1 in flight 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.894+0000 7f70db65b640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x562b42e092e0 0x562b42e0b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:30.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.894+0000 7f70db65b640 10 monclient: get_auth_request con 0x562b42e092e0 auth_method 0 2026-02-24T18:35:30.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.896+0000 7f70db65b640 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x562b42e092e0 0x562b42e0b770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:30.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.896+0000 7f70c37fe640 10 client.4386.objecter ms_handle_connect 0x562b42e092e0 2026-02-24T18:35:30.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 <== osd.3 v2:10.20.193.112:6826/230320724 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f70d0099a30 con 0x562b42e092e0 2026-02-24T18:35:30.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-24T18:35:30.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 10 client.4386.objecter in handle_osd_op_reply 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 7 client.4386.objecter handle_osd_op_reply 2 ondisk uv 0 in 2.2 attempt 0 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 10 client.4386.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 10 client.4386.objecter op 0 rval 0 len 0 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 15 client.4386.objecter handle_osd_op_reply completed tid 2 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 15 client.4386.objecter _finish_op 2 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 15 client.4386.objecter _session_op_remove 3 2 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70db65b640 5 client.4386.objecter 0 in flight 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 10 client.4386.objecter _op_submit op 0x562b42e07910 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 20 client.4386.objecter _calc_target epoch 26 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 20 client.4386.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-24T18:35:30.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 10 client.4386.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 20 client.4386.objecter _get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 10 client.4386.objecter _op_submit oid rbd_info '@2' '@2' [read 0~0] tid 3 osd.3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 20 client.4386.objecter get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 15 client.4386.objecter _session_op_assign 3 3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 15 client.4386.objecter _send_op 3 to 2.2 on osd.3 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26) -- 0x562b42e07dc0 con 0x562b42e092e0 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:30.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.946+0000 7f70dcb950c0 5 client.4386.objecter 1 in flight 2026-02-24T18:35:30.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 <== osd.3 v2:10.20.193.112:6826/230320724 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f70d0099a30 con 0x562b42e092e0 2026-02-24T18:35:30.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-24T18:35:30.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 10 client.4386.objecter in handle_osd_op_reply 2026-02-24T18:35:30.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 7 client.4386.objecter handle_osd_op_reply 3 ondisk uv 0 in 2.2 attempt 0 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 10 client.4386.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 10 client.4386.objecter op 0 rval 0 len 0 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 15 client.4386.objecter handle_osd_op_reply completed tid 3 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 15 client.4386.objecter _finish_op 3 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 15 client.4386.objecter _session_op_remove 3 3 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70db65b640 5 client.4386.objecter 0 in flight 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70d9657640 10 client.4386.objecter allocate_selfmanaged_snap; pool: 2 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70d9657640 10 client.4386.objecter pool_op_submit 4 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70d9657640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:30.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:30.955+0000 7f70d9657640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f70b80030d0 con 0x562b42d09b60 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 10 ==== pool_op_reply(tid 4 (0) Success v27) ==== 55+0+0 (secure 0 0 0) 0x7f70d006a000 con 0x562b42d09b60 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 pool_op_reply(tid 4 (0) Success v27) 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 client.4386.objecter handle_pool_op_reply pool_op_reply(tid 4 (0) Success v27) 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 client.4386.objecter have request 4 at 0x7f70b8003060 Op: create unmanaged snap 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 20 client.4386.objecter waiting for client to reach epoch 27 before calling back 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 client.4386.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 monclient: _renew_subs 2026-02-24T18:35:31.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:31.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=27}) -- 0x7f70bc07f530 con 0x562b42d09b60 2026-02-24T18:35:31.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.339+0000 7f70c37fe640 10 client.4386.objecter done 2026-02-24T18:35:31.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 11 ==== osd_map(27..27 src has 1..27) ==== 626+0+0 (secure 0 0 0) 0x7f70d0069040 con 0x562b42d09b60 2026-02-24T18:35:31.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_map(27..27 src has 1..27) 2026-02-24T18:35:31.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map got epochs [27,27] > 26 2026-02-24T18:35:31.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map decoding incremental epoch 27 2026-02-24T18:35:31.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70c37fe640 20 client.4386.objecter dump_active .. 0 homeless 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 librados: queue_aio_write 0x562b42e03cc0 completion 0x562b42b6f5f0 write_seq 1 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 10 client.4386.objecter _op_submit op 0x7f70b4006380 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 client.4386.objecter _calc_target epoch 27 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 client.4386.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 10 client.4386.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 client.4386.objecter _get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 10 client.4386.objecter _op_submit oid rbd_info '@2' '@2' [create,write 0~8 in=8b] tid 5 osd.3 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 client.4386.objecter get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 15 client.4386.objecter _session_op_assign 3 5 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 15 client.4386.objecter _send_op 5 to 2.2 on osd.3 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x7f70b4006a20 con 0x562b42e092e0 2026-02-24T18:35:31.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:31.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.343+0000 7f70d9e58640 5 client.4386.objecter 1 in flight 2026-02-24T18:35:31.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 <== osd.3 v2:10.20.193.112:6826/230320724 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v27'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f70d0099a30 con 0x562b42e092e0 2026-02-24T18:35:31.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_op_reply(5 rbd_info [create,write 0~8] v27'1 uv1 ondisk = 0) 2026-02-24T18:35:31.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 10 client.4386.objecter in handle_osd_op_reply 2026-02-24T18:35:31.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 7 client.4386.objecter handle_osd_op_reply 5 ondisk uv 1 in 2.2 attempt 0 2026-02-24T18:35:31.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 10 client.4386.objecter op 0 rval 0 len 0 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 10 client.4386.objecter op 1 rval 0 len 0 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 15 client.4386.objecter handle_osd_op_reply completed tid 5 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 15 client.4386.objecter _finish_op 5 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 15 client.4386.objecter _session_op_remove 3 5 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 5 client.4386.objecter 0 in flight 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70db65b640 20 librados: complete_aio_write 0x562b42b6f5f0 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70d9657640 10 client.4386.objecter delete_selfmanaged_snap; pool: 2; snap: 2 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70d9657640 10 client.4386.objecter pool_op_submit 6 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70d9657640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:31.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:31.613+0000 7f70d9657640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v27) -- 0x7f70b8004520 con 0x562b42d09b60 2026-02-24T18:35:32.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.253+0000 7f70c27fc640 10 monclient: tick 2026-02-24T18:35:32.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.253+0000 7f70c27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:32.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.253+0000 7f70c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:32.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.253+0000 7f70c27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:32.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 12 ==== pool_op_reply(tid 6 (0) Success v28) ==== 43+0+0 (secure 0 0 0) 0x7f70d0060dc0 con 0x562b42d09b60 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 pool_op_reply(tid 6 (0) Success v28) 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 client.4386.objecter handle_pool_op_reply pool_op_reply(tid 6 (0) Success v28) 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 client.4386.objecter have request 6 at 0x7f70b8003060 Op: delete unmanaged snap 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 20 client.4386.objecter waiting for client to reach epoch 28 before calling back 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 client.4386.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 monclient: _renew_subs 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=28}) -- 0x7f70bc07f700 con 0x562b42d09b60 2026-02-24T18:35:32.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.415+0000 7f70c37fe640 10 client.4386.objecter done 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70c37fe640 1 -- 10.20.193.18:0/3018835393 <== mon.0 v2:10.20.193.18:3300/0 13 ==== osd_map(28..28 src has 1..28) ==== 654+0+0 (secure 0 0 0) 0x7f70d0083230 con 0x562b42d09b60 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70c37fe640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_map(28..28 src has 1..28) 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map got epochs [28,28] > 27 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70c37fe640 3 client.4386.objecter handle_osd_map decoding incremental epoch 28 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70c37fe640 20 client.4386.objecter dump_active .. 0 homeless 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 librados: queue_aio_write 0x562b42e03cc0 completion 0x7f70b8001d60 write_seq 2 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 10 client.4386.objecter _op_submit op 0x7f70b4006380 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 client.4386.objecter _calc_target epoch 28 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-24T18:35:32.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 client.4386.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 10 client.4386.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 client.4386.objecter _get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 10 client.4386.objecter _op_submit oid rbd_info '@2' '@2' [write 0~19 in=19b] tid 7 osd.3 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 client.4386.objecter get_session s=0x562b42e08c60 osd=3 3 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 15 client.4386.objecter _session_op_assign 3 7 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 15 client.4386.objecter _send_op 7 to 2.2 on osd.3 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 1 -- 10.20.193.18:0/3018835393 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) -- 0x7f70b4006dc0 con 0x562b42e092e0 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:32.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.419+0000 7f70d9e58640 5 client.4386.objecter 1 in flight 2026-02-24T18:35:32.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 1 -- 10.20.193.18:0/3018835393 <== osd.3 v2:10.20.193.112:6826/230320724 4 ==== osd_op_reply(7 rbd_info [write 0~19] v28'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f70d0099a30 con 0x562b42e092e0 2026-02-24T18:35:32.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 10 client.4386.objecter ms_dispatch2 0x562b42bc69a0 osd_op_reply(7 rbd_info [write 0~19] v28'2 uv2 ondisk = 0) 2026-02-24T18:35:32.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 10 client.4386.objecter in handle_osd_op_reply 2026-02-24T18:35:32.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 7 client.4386.objecter handle_osd_op_reply 7 ondisk uv 2 in 2.2 attempt 0 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 10 client.4386.objecter op 0 rval 0 len 0 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 15 client.4386.objecter handle_osd_op_reply completed tid 7 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 15 client.4386.objecter _finish_op 7 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 4 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 15 client.4386.objecter _session_op_remove 3 7 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 5 client.4386.objecter 0 in flight 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70db65b640 20 librados: complete_aio_write 0x7f70b8001d60 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 librados: watch_flush enter 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 librados: watch_flush exit 2026-02-24T18:35:32.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 client.4386.objecter close_session for osd.3 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x562b42e092e0 msgr2=0x562b42e0b770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x562b42e092e0 0x562b42e0b770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 20 client.4386.objecter put_session s=0x562b42e08c60 osd=3 1 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 client.4386.objecter close_session for osd.4 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x562b42cc78e0 msgr2=0x562b42daba80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x562b42cc78e0 0x562b42daba80 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 20 client.4386.objecter put_session s=0x562b42b4b230 osd=4 1 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 20 client.4386.objecter shutdown clearing up homeless session... 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 client.4386.objecter successfully canceled tick 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70bc076da0 msgr2=0x7f70bc079260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70bc076da0 0x7f70bc079260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70c406a000 tx=0x7f70c4064000 comp rx=0 tx=0).stop 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 10 monclient: shutdown 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 msgr2=0x562b42d8ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70d0008810 tx=0x7f70d0008910 comp rx=0 tx=0).stop 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.521+0000 7f70dc8e4640 1 -- 10.20.193.18:0/3018835393 reap_dead start 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 shutdown_connections 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 --2- 10.20.193.18:0/3018835393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x562b42d09b60 0x562b42d8ccf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 >> 10.20.193.18:0/3018835393 conn(0x562b42cc8730 msgr2=0x562b42d93e40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 shutdown_connections 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 -- 10.20.193.18:0/3018835393 wait complete. 2026-02-24T18:35:32.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.522+0000 7f70dcb950c0 1 librados: shutdown 2026-02-24T18:35:32.530 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-24T18:35:32.531 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-24T18:35:32.531 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-24T18:35:32.531 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-24T18:35:32.531 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-24T18:35:32.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.636+0000 7f6e2d216640 10 monclient: get_monmap_and_config 2026-02-24T18:35:32.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.636+0000 7f6e2d216640 10 monclient: build_initial_monmap 2026-02-24T18:35:32.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.636+0000 7f6e2d216640 10 monclient: monmap: 2026-02-24T18:35:32.638 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:32.637691+0000 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:32.637691+0000 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:32.639 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:32.640 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 Processor -- start 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 -- start start 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient: init 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient: _add_conns ranks=[3,2,1,4,5,0] 2026-02-24T18:35:32.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6e281527d0 0x7f6e2806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-a con 0x7f6e281527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e28152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6e280c6190 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2806e300 con 0x7f6e280c6190 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2806e5e0 con 0x7f6e28152bb0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2806f1f0 con 0x7f6e281527d0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.637+0000 7f6e2d216640 10 monclient(hunting): authenticate will time out at 1476.318933s 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6e281527d0 0x7f6e2806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:32.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e27577640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f6e280c6190 0x7f6e280c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51116/0 (socket says 10.20.193.18:51116) 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e27577640 1 -- 10.20.193.18:0/629343999 learned_addr learned my addr 10.20.193.18:0/629343999 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): get_auth_request con 0x7f6e281527d0 auth_method 0 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.638+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 10 monclient(hunting): handle_auth_done global_id 4395 payload 274 2026-02-24T18:35:32.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 1 -- 10.20.193.18:0/629343999 >> v1:10.20.193.110:6789/0 conn(0x7f6e28152bb0 legacy=0x7f6e280c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 1 -- 10.20.193.18:0/629343999 >> v1:10.20.193.32:6789/0 conn(0x7f6e280c6190 legacy=0x7f6e280c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 1 monclient: found mon.noname-a 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.639+0000 7f6e26d76640 1 -- 10.20.193.18:0/629343999 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2806e8c0 con 0x7f6e281527d0 2026-02-24T18:35:32.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.640+0000 7f6e26d76640 1 --2- 10.20.193.18:0/629343999 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e281527d0 0x7f6e2806da60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e2806f370 tx=0x7f6e1c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db009158995813 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:32.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 1 -- 10.20.193.18:0/629343999 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c03c070 con 0x7f6e281527d0 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: dump: 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.653 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: _finish_auth 0 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: _check_auth_tickets 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 1 -- 10.20.193.18:0/629343999 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c02fb70 con 0x7f6e281527d0 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 1 -- 10.20.193.18:0/629343999 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c037440 con 0x7f6e281527d0 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 5 monclient: authenticate success, global_id 4395 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:32.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 4 set_mon_vals no callback set 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e25d74640 10 monclient: dump: 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:32.655 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 10 monclient: shutdown 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 1 -- 10.20.193.18:0/629343999 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e281527d0 msgr2=0x7f6e2806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 1 --2- 10.20.193.18:0/629343999 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e281527d0 0x7f6e2806da60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e2806f370 tx=0x7f6e1c02ee60 comp rx=0 tx=0).stop 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 1 -- 10.20.193.18:0/629343999 shutdown_connections 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.642+0000 7f6e2d216640 1 --2- 10.20.193.18:0/629343999 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e281527d0 0x7f6e2806da60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 -- 10.20.193.18:0/629343999 >> 10.20.193.18:0/629343999 conn(0x7f6e280cd780 msgr2=0x7f6e280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 -- 10.20.193.18:0/629343999 shutdown_connections 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 -- 10.20.193.18:0/629343999 wait complete. 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 10 monclient: build_initial_monmap 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 10 monclient: monmap: 2026-02-24T18:35:32.656 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 librados: starting msgr at 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 librados: starting objecter 2026-02-24T18:35:32.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 Processor -- start 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 -- start start 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 librados: setting wanted keys 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 1 librados: calling monclient init 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.643+0000 7f6e2d216640 10 monclient: init 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e280c6190 0x7f6e281f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-b con 0x7f6e280c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e281527d0 0x7f6e280735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-c con 0x7f6e281527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): picked mon.noname-a con 0x7f6e28152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:32.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2806dfa0 con 0x7f6e28152bb0 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2806ee80 con 0x7f6e280c6190 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280c9570 con 0x7f6e281527d0 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e2d216640 10 monclient(hunting): authenticate will time out at 1476.325357s 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e280c6190 0x7f6e281f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e281527d0 0x7f6e280735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41084/0 (socket says 10.20.193.18:41084) 2026-02-24T18:35:32.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 1 -- 10.20.193.18:0/720390076 learned_addr learned my addr 10.20.193.18:0/720390076 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): get_auth_request con 0x7f6e28152bb0 auth_method 0 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26d76640 10 monclient(hunting): get_auth_request con 0x7f6e280c6190 auth_method 0 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26575640 10 monclient(hunting): get_auth_request con 0x7f6e281527d0 auth_method 0 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:32.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e26575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.644+0000 7f6e27577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.645+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 -- 10.20.193.18:0/720390076 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e281527d0 msgr2=0x7f6e280735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e281527d0 0x7f6e280735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 -- 10.20.193.18:0/720390076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e280c6190 msgr2=0x7f6e281f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e280c6190 0x7f6e281f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 10 monclient: _finish_hunting 0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 monclient: found mon.noname-a 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.646+0000 7f6e27577640 1 -- 10.20.193.18:0/720390076 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e28072520 con 0x7f6e28152bb0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.647+0000 7f6e27577640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800cb50 tx=0x7f6e1800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.647+0000 7f6e077fe640 10 client.?.objecter ms_handle_connect 0x7f6e28152bb0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.647+0000 7f6e077fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18017070 con 0x7f6e28152bb0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:32.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: dump: 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1800ad50 con 0x7f6e28152bb0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18012600 con 0x7f6e28152bb0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 5 monclient: authenticate success, global_id 4398 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _renew_subs 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e25573640 4 set_mon_vals no callback set 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e28072800 con 0x7f6e28152bb0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:32.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e077fe640 10 monclient: dump: 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _renew_subs 2026-02-24T18:35:32.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e28072dc0 con 0x7f6e28152bb0 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 1 librados: init done 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.649+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2806da60 con 0x7f6e28152bb0 2026-02-24T18:35:32.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.661+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6e1801b3b0 con 0x7f6e28152bb0 2026-02-24T18:35:32.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.661+0000 7f6e077fe640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 0x7f6df407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.661+0000 7f6e26d76640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 0x7f6df407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.661+0000 7f6e26d76640 10 monclient: get_auth_request con 0x7f6df4078e50 auth_method 0 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.662+0000 7f6e26d76640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 0x7f6df407b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c039000 tx=0x7f6e1c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.662+0000 7f6e077fe640 10 client.4398.objecter ms_handle_connect 0x7f6df4078e50 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.663+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f6e1805db60 con 0x7f6e28152bb0 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.663+0000 7f6e077fe640 10 client.4398.objecter ms_dispatch2 0x7f6e28000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.663+0000 7f6e077fe640 3 client.4398.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.663+0000 7f6e077fe640 3 client.4398.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:32.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.663+0000 7f6e077fe640 20 client.4398.objecter dump_active .. 0 homeless 2026-02-24T18:35:32.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.762+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e18022080 con 0x7f6e28152bb0 2026-02-24T18:35:32.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.762+0000 7f6e077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:32.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.762+0000 7f6e077fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:32.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.871+0000 7f6e2d216640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-24T18:35:32.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.871+0000 7f6e2d216640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-24T18:35:32.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.871+0000 7f6e2d216640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:32.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.871+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f6e280cb310 con 0x7f6e28152bb0 2026-02-24T18:35:32.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.941+0000 7f6e077fe640 1 -- 10.20.193.18:0/720390076 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+191894 (secure 0 0 0) 0x7f6e18061b20 con 0x7f6e28152bb0 2026-02-24T18:35:32.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.941+0000 7f6e077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-24T18:35:32.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.941+0000 7f6e077fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:32.943 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 10 librados: watch_flush enter 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 10 librados: watch_flush exit 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 20 client.4398.objecter shutdown clearing up homeless session... 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 10 client.4398.objecter successfully canceled tick 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 msgr2=0x7f6df407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 0x7f6df407b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c039000 tx=0x7f6e1c033000 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 10 monclient: shutdown 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 msgr2=0x7f6e28073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800cb50 tx=0x7f6e1800a430 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 shutdown_connections 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6df4078e50 0x7f6df407b310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6e28152bb0 0x7f6e28073b00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6e281527d0 0x7f6e280735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 --2- 10.20.193.18:0/720390076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6e280c6190 0x7f6e281f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 >> 10.20.193.18:0/720390076 conn(0x7f6e280cd780 msgr2=0x7f6e280c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 shutdown_connections 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 -- 10.20.193.18:0/720390076 wait complete. 2026-02-24T18:35:32.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:32.943+0000 7f6e2d216640 1 librados: shutdown 2026-02-24T18:35:32.953 INFO:teuthology.orchestra.run.trial018.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6800","nonce":3643511854},{"type":"v1","addr":"10.20.193.112:6801","nonce":3643511854}]},"active_addr":"10.20.193.112:6801/3643511854","active_change":"2026-02-24T18:33:47.701761+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.112:0","nonce":526819041}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.112:0","nonce":3049471541}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.112:0","nonce":2754360817}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.112:0","nonce":598859222}]}]} 2026-02-24T18:35:32.956 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-24T18:35:32.956 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-24T18:35:32.956 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:33.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 10 monclient: monmap: 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.061113+0000 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.061113+0000 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.062 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 1 Processor -- start 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 1 -- start start 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.060+0000 7f07457a9640 10 monclient: init 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient: _add_conns ranks=[4,3,0,5,1,2] 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0740155f80 0x7f074015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0740155f80 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f07400ccb40 0x7f074015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-a con 0x7f07400ccb40 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0740156360 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400d0380 con 0x7f0740156360 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f074006d5b0 con 0x7f07400ccb40 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f074006c9a0 con 0x7f0740155f80 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f07457a9640 10 monclient(hunting): authenticate will time out at 1476.742313s 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073e7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f07400ccb40 0x7f074015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073f7fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f0740156360 0x7f074015f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58496/0 (socket says 10.20.193.18:58496) 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073f7fe640 1 -- 10.20.193.18:0/3487621156 learned_addr learned my addr 10.20.193.18:0/3487621156 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073e7fc640 10 monclient(hunting): get_auth_request con 0x7f07400ccb40 auth_method 0 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.061+0000 7f073e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 1 --2- 10.20.193.18:0/3487621156 >> v2:10.20.193.32:3300/0 conn(0x7f0740155f80 0x7f074015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): get_auth_request con 0x7f0740155f80 auth_method 0 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.062+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 10 monclient(hunting): handle_auth_done global_id 4401 payload 274 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 1 -- 10.20.193.18:0/3487621156 >> v2:10.20.193.32:3300/0 conn(0x7f0740155f80 msgr2=0x7f074015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 1 --2- 10.20.193.18:0/3487621156 >> v2:10.20.193.32:3300/0 conn(0x7f0740155f80 0x7f074015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 1 -- 10.20.193.18:0/3487621156 >> v1:10.20.193.18:6789/0 conn(0x7f0740156360 legacy=0x7f074015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.063+0000 7f073e7fc640 1 -- 10.20.193.18:0/3487621156 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074006cc80 con 0x7f07400ccb40 2026-02-24T18:35:33.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.065+0000 7f073e7fc640 1 --2- 10.20.193.18:0/3487621156 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f074015d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f074006f270 tx=0x7f072802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa4c2da23589d3e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 1 -- 10.20.193.18:0/3487621156 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072803c070 con 0x7f07400ccb40 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: dump: 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 1 -- 10.20.193.18:0/3487621156 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072802fa60 con 0x7f07400ccb40 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f07457a9640 5 monclient: authenticate success, global_id 4401 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 1 -- 10.20.193.18:0/3487621156 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072802fe30 con 0x7f07400ccb40 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f073dffb640 10 monclient: dump: 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f07457a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.067+0000 7f07457a9640 4 set_mon_vals no callback set 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 10 monclient: shutdown 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 -- 10.20.193.18:0/3487621156 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 msgr2=0x7f074015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 --2- 10.20.193.18:0/3487621156 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f074015d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f074006f270 tx=0x7f072802ed50 comp rx=0 tx=0).stop 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 -- 10.20.193.18:0/3487621156 shutdown_connections 2026-02-24T18:35:33.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 --2- 10.20.193.18:0/3487621156 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f074015d320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 --2- 10.20.193.18:0/3487621156 >> v2:10.20.193.32:3300/0 conn(0x7f0740155f80 0x7f074015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 -- 10.20.193.18:0/3487621156 >> 10.20.193.18:0/3487621156 conn(0x7f07400cc020 msgr2=0x7f07400cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 -- 10.20.193.18:0/3487621156 shutdown_connections 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 1 -- 10.20.193.18:0/3487621156 wait complete. 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.068+0000 7f07457a9640 10 monclient: monmap: 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 librados: starting msgr at 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 librados: starting objecter 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 Processor -- start 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 -- start start 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 librados: setting wanted keys 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 librados: calling monclient init 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient: init 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-a con 0x7f07400ccb40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0740155f80 0x7f074014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0740155f80 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0740156360 0x7f074006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): picked mon.noname-c con 0x7f0740156360 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0740072eb0 con 0x7f07400ccb40 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f074006d240 con 0x7f0740155f80 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f074015d860 con 0x7f0740156360 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f07457a9640 10 monclient(hunting): authenticate will time out at 1476.750655s 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.069+0000 7f073effd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41110/0 (socket says 10.20.193.18:41110) 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 1 -- 10.20.193.18:0/3640241908 learned_addr learned my addr 10.20.193.18:0/3640241908 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): get_auth_request con 0x7f07400ccb40 auth_method 0 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0740155f80 0x7f074014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0740156360 0x7f074006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): get_auth_request con 0x7f0740155f80 auth_method 0 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): get_auth_request con 0x7f0740156360 auth_method 0 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.070+0000 7f073f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 10 monclient(hunting): handle_auth_done global_id 4407 payload 824 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 -- 10.20.193.18:0/3640241908 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0740156360 msgr2=0x7f074006f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0740156360 0x7f074006f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 -- 10.20.193.18:0/3640241908 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0740155f80 msgr2=0x7f074014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0740155f80 0x7f074014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.071+0000 7f073effd640 1 -- 10.20.193.18:0/3640241908 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074014df50 con 0x7f07400ccb40 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.072+0000 7f073effd640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f073400ebe0 tx=0x7f073400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.072+0000 7f071f7fe640 10 client.?.objecter ms_handle_connect 0x7f07400ccb40 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.072+0000 7f071f7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734017070 con 0x7f07400ccb40 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: dump: 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073400cc40 con 0x7f07400ccb40 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734013470 con 0x7f07400ccb40 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 5 monclient: authenticate success, global_id 4407 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 10 monclient: _renew_subs 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f073d7fa640 4 set_mon_vals no callback set 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0740150ff0 con 0x7f07400ccb40 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f071f7fe640 10 monclient: dump: 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 10 client.4407.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 10 monclient: _renew_subs 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401515b0 con 0x7f07400ccb40 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.074+0000 7f07457a9640 1 librados: init done 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.075+0000 7f07457a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.075+0000 7f07457a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.075+0000 7f07457a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.075+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074006f5b0 con 0x7f07400ccb40 2026-02-24T18:35:33.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.180+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0734024060 con 0x7f07400ccb40 2026-02-24T18:35:33.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.180+0000 7f071f7fe640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 0x7f070c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.180+0000 7f073e7fc640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 0x7f070c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.181+0000 7f073e7fc640 10 monclient: get_auth_request con 0x7f070c078e00 auth_method 0 2026-02-24T18:35:33.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.181+0000 7f073e7fc640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 0x7f070c07b2c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0728039000 tx=0x7f0728033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.181+0000 7f071f7fe640 10 client.4407.objecter ms_handle_connect 0x7f070c078e00 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f073405e290 con 0x7f07400ccb40 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 10 client.4407.objecter ms_dispatch2 0x7f0740000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 3 client.4407.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 3 client.4407.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 20 client.4407.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f074014df50 con 0x7f07400ccb40 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.188+0000 7f071f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:33.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.271+0000 7f07457a9640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.271+0000 7f07457a9640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.271+0000 7f07457a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.271+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f074015dbf0 con 0x7f07400ccb40 2026-02-24T18:35:33.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.322+0000 7f071f7fe640 1 -- 10.20.193.18:0/3640241908 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+24707 (secure 0 0 0) 0x7f0734062250 con 0x7f07400ccb40 2026-02-24T18:35:33.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.322+0000 7f071f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.322+0000 7f071f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:33.324 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:33.324 INFO:teuthology.orchestra.run.trial018.stdout:{"epoch":28,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:35:32.309762+0000","last_up_change":"2026-02-24T18:35:26.937667+0000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:35:26.061331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:35:29.058214+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6817","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6819","nonce":1076077393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6821","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6823","nonce":1076077393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6829","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6831","nonce":1076077393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6825","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6827","nonce":1076077393}]},"public_addr":"10.20.193.18:6819/1076077393","cluster_addr":"10.20.193.18:6823/1076077393","heartbeat_back_addr":"10.20.193.18:6831/1076077393","heartbeat_front_addr":"10.20.193.18:6827/1076077393","state":["exists","up"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6800","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6801","nonce":2795974188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6802","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6803","nonce":2795974188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6806","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6807","nonce":2795974188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6804","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6805","nonce":2795974188}]},"public_addr":"10.20.193.32:6801/2795974188","cluster_addr":"10.20.193.32:6803/2795974188","heartbeat_back_addr":"10.20.193.32:6807/2795974188","heartbeat_front_addr":"10.20.193.32:6805/2795974188","state":["exists","up"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6825","nonce":3650974883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6826","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6827","nonce":3650974883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6830","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6831","nonce":3650974883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6829","nonce":3650974883}]},"public_addr":"10.20.193.110:6825/3650974883","cluster_addr":"10.20.193.110:6827/3650974883","heartbeat_back_addr":"10.20.193.110:6831/3650974883","heartbeat_front_addr":"10.20.193.110:6829/3650974883","state":["exists","up"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6826","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6827","nonce":230320724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6828","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6829","nonce":230320724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6832","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6833","nonce":230320724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6830","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6831","nonce":230320724}]},"public_addr":"10.20.193.112:6827/230320724","cluster_addr":"10.20.193.112:6829/230320724","heartbeat_back_addr":"10.20.193.112:6833/230320724","heartbeat_front_addr":"10.20.193.112:6831/230320724","state":["exists","up"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6800","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6801","nonce":3481454300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6802","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6803","nonce":3481454300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6806","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6807","nonce":3481454300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6804","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6805","nonce":3481454300}]},"public_addr":"10.20.193.18:6801/3481454300","cluster_addr":"10.20.193.18:6803/3481454300","heartbeat_back_addr":"10.20.193.18:6807/3481454300","heartbeat_front_addr":"10.20.193.18:6805/3481454300","state":["exists","up"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6808","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6809","nonce":2128834550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6810","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6811","nonce":2128834550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6814","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6815","nonce":2128834550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6812","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6813","nonce":2128834550}]},"public_addr":"10.20.193.32:6809/2128834550","cluster_addr":"10.20.193.32:6811/2128834550","heartbeat_back_addr":"10.20.193.32:6815/2128834550","heartbeat_front_addr":"10.20.193.32:6813/2128834550","state":["exists","up"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6801","nonce":3466222224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6803","nonce":3466222224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6807","nonce":3466222224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6805","nonce":3466222224}]},"public_addr":"10.20.193.110:6801/3466222224","cluster_addr":"10.20.193.110:6803/3466222224","heartbeat_back_addr":"10.20.193.110:6807/3466222224","heartbeat_front_addr":"10.20.193.110:6805/3466222224","state":["exists","up"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6810","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6811","nonce":92089351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6814","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6815","nonce":92089351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6821","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6823","nonce":92089351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6817","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6819","nonce":92089351}]},"public_addr":"10.20.193.112:6811/92089351","cluster_addr":"10.20.193.112:6815/92089351","heartbeat_back_addr":"10.20.193.112:6823/92089351","heartbeat_front_addr":"10.20.193.112:6819/92089351","state":["exists","up"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6808","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6809","nonce":1977979333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6810","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6811","nonce":1977979333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6814","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6815","nonce":1977979333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6812","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6813","nonce":1977979333}]},"public_addr":"10.20.193.18:6809/1977979333","cluster_addr":"10.20.193.18:6811/1977979333","heartbeat_back_addr":"10.20.193.18:6815/1977979333","heartbeat_front_addr":"10.20.193.18:6813/1977979333","state":["exists","up"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6816","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6817","nonce":1806399530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6818","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6819","nonce":1806399530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6822","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6823","nonce":1806399530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6820","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6821","nonce":1806399530}]},"public_addr":"10.20.193.32:6817/1806399530","cluster_addr":"10.20.193.32:6819/1806399530","heartbeat_back_addr":"10.20.193.32:6823/1806399530","heartbeat_front_addr":"10.20.193.32:6821/1806399530","state":["exists","up"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6809","nonce":4149932150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6811","nonce":4149932150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6815","nonce":4149932150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6813","nonce":4149932150}]},"public_addr":"10.20.193.110:6809/4149932150","cluster_addr":"10.20.193.110:6811/4149932150","heartbeat_back_addr":"10.20.193.110:6815/4149932150","heartbeat_front_addr":"10.20.193.110:6813/4149932150","state":["exists","up"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6812","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6813","nonce":2071089960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6816","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6818","nonce":2071089960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6824","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6825","nonce":2071089960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6820","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6822","nonce":2071089960}]},"public_addr":"10.20.193.112:6813/2071089960","cluster_addr":"10.20.193.112:6818/2071089960","heartbeat_back_addr":"10.20.193.112:6825/2071089960","heartbeat_front_addr":"10.20.193.112:6822/2071089960","state":["exists","up"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6816","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6818","nonce":392025864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6820","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6822","nonce":392025864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6828","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6830","nonce":392025864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6824","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6826","nonce":392025864}]},"public_addr":"10.20.193.18:6818/392025864","cluster_addr":"10.20.193.18:6822/392025864","heartbeat_back_addr":"10.20.193.18:6830/392025864","heartbeat_front_addr":"10.20.193.18:6826/392025864","state":["exists","up"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6824","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6825","nonce":316721302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6826","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6827","nonce":316721302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6830","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6831","nonce":316721302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6828","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6829","nonce":316721302}]},"public_addr":"10.20.193.32:6825/316721302","cluster_addr":"10.20.193.32:6827/316721302","heartbeat_back_addr":"10.20.193.32:6831/316721302","heartbeat_front_addr":"10.20.193.32:6829/316721302","state":["exists","up"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6817","nonce":3577843273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6818","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6819","nonce":3577843273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6822","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6823","nonce":3577843273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6821","nonce":3577843273}]},"public_addr":"10.20.193.110:6817/3577843273","cluster_addr":"10.20.193.110:6819/3577843273","heartbeat_back_addr":"10.20.193.110:6823/3577843273","heartbeat_front_addr":"10.20.193.110:6821/3577843273","state":["exists","up"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6802","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6803","nonce":4220209204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6804","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6805","nonce":4220209204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6808","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6809","nonce":4220209204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6806","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6807","nonce":4220209204}]},"public_addr":"10.20.193.112:6803/4220209204","cluster_addr":"10.20.193.112:6805/4220209204","heartbeat_back_addr":"10.20.193.112:6809/4220209204","heartbeat_front_addr":"10.20.193.112:6807/4220209204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.022230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.519970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.248082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.777136+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.483378+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.749510+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.414866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.123257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.766447+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.874254+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.504376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.120504+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.013826+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.378335+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.904134+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.610284+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 10 librados: watch_flush enter 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 10 librados: watch_flush exit 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 20 client.4407.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 10 client.4407.objecter successfully canceled tick 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 msgr2=0x7f070c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 0x7f070c07b2c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0728039000 tx=0x7f0728033000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 10 monclient: shutdown 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 msgr2=0x7f0740150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f073400ebe0 tx=0x7f073400c520 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 shutdown_connections 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.324+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f070c078e00 0x7f070c07b2c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0740155f80 0x7f074014ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0740156360 0x7f074006f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 --2- 10.20.193.18:0/3640241908 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07400ccb40 0x7f0740150940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 >> 10.20.193.18:0/3640241908 conn(0x7f07400cc020 msgr2=0x7f074015de90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 shutdown_connections 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 -- 10.20.193.18:0/3640241908 wait complete. 2026-02-24T18:35:33.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.325+0000 7f07457a9640 1 librados: shutdown 2026-02-24T18:35:33.336 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-24T18:35:33.336 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.403+0000 7f4adc934640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.403+0000 7f4adc934640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.403+0000 7f4adc934640 10 monclient: monmap: 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.404477+0000 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.404477+0000 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.405 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 Processor -- start 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 -- start start 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient: init 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient: _add_conns ranks=[4,0,5,2,1,3] 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4ad406ff20 0x7f4ad4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-b con 0x7f4ad406ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4ad40c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4ad415a5e0 0x7f4ad414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-c con 0x7f4ad415a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad40c7e90 con 0x7f4ad40c99a0 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad4159a40 con 0x7f4ad406ff20 2026-02-24T18:35:33.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad40c8170 con 0x7f4ad415a5e0 2026-02-24T18:35:33.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.404+0000 7f4adc934640 10 monclient(hunting): authenticate will time out at 1477.085638s 2026-02-24T18:35:33.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4ad406ff20 0x7f4ad4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4ad406ff20 0x7f4ad4070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52946/0 (socket says 10.20.193.18:52946) 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 1 -- 10.20.193.18:0/3634346079 learned_addr learned my addr 10.20.193.18:0/3634346079 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 10 monclient(hunting): get_auth_request con 0x7f4ad406ff20 auth_method 0 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4ada6a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.419+0000 7f4adaeaa640 1 --2- 10.20.193.18:0/3634346079 >> v2:10.20.193.110:3300/0 conn(0x7f4ad415a5e0 0x7f4ad414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): get_auth_request con 0x7f4ad415a5e0 auth_method 0 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.420+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_done global_id 4399 payload 274 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 1 -- 10.20.193.18:0/3634346079 >> v2:10.20.193.110:3300/0 conn(0x7f4ad415a5e0 msgr2=0x7f4ad414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 1 --2- 10.20.193.18:0/3634346079 >> v2:10.20.193.110:3300/0 conn(0x7f4ad415a5e0 0x7f4ad414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 1 -- 10.20.193.18:0/3634346079 >> v1:10.20.193.18:6789/0 conn(0x7f4ad40c99a0 legacy=0x7f4ad4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.421+0000 7f4ada6a9640 1 -- 10.20.193.18:0/3634346079 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad40c8450 con 0x7f4ad406ff20 2026-02-24T18:35:33.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.422+0000 7f4ada6a9640 1 --2- 10.20.193.18:0/3634346079 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad4070300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009970 tx=0x7f4acc02eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931aa99daa830276 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 1 -- 10.20.193.18:0/3634346079 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc03c070 con 0x7f4ad406ff20 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 10 monclient: dump: 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4ad96a7640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4adc934640 5 monclient: authenticate success, global_id 4399 2026-02-24T18:35:33.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.423+0000 7f4adc934640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 1 -- 10.20.193.18:0/3634346079 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc02fb20 con 0x7f4ad406ff20 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 1 -- 10.20.193.18:0/3634346079 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc037440 con 0x7f4ad406ff20 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4ad96a7640 10 monclient: dump: 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.426 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 4 set_mon_vals no callback set 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 10 monclient: shutdown 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 -- 10.20.193.18:0/3634346079 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 msgr2=0x7f4ad4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 --2- 10.20.193.18:0/3634346079 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad4070300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009970 tx=0x7f4acc02eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 -- 10.20.193.18:0/3634346079 shutdown_connections 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 --2- 10.20.193.18:0/3634346079 >> v2:10.20.193.110:3300/0 conn(0x7f4ad415a5e0 0x7f4ad414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 --2- 10.20.193.18:0/3634346079 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad4070300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 -- 10.20.193.18:0/3634346079 >> 10.20.193.18:0/3634346079 conn(0x7f4ad40cd780 msgr2=0x7f4ad40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 -- 10.20.193.18:0/3634346079 shutdown_connections 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.424+0000 7f4adc934640 1 -- 10.20.193.18:0/3634346079 wait complete. 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient: monmap: 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.427 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.428 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.428 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.428 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.428 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 librados: starting msgr at 2026-02-24T18:35:33.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 librados: starting objecter 2026-02-24T18:35:33.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 Processor -- start 2026-02-24T18:35:33.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 -- start start 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 librados: setting wanted keys 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 librados: calling monclient init 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient: init 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad41f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-b con 0x7f4ad406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 0x7f4ad40bf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-a con 0x7f4ad40c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ad415a5e0 0x7f4ad414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): picked mon.noname-c con 0x7f4ad415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4150350 con 0x7f4ad40c99a0 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad40c8a10 con 0x7f4ad406ff20 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41504d0 con 0x7f4ad415a5e0 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.425+0000 7f4adc934640 10 monclient(hunting): authenticate will time out at 1477.106940s 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 0x7f4ad40bf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad41f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ad415a5e0 0x7f4ad414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 0x7f4ad40bf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41114/0 (socket says 10.20.193.18:41114) 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 1 -- 10.20.193.18:0/1458983517 learned_addr learned my addr 10.20.193.18:0/1458983517 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): get_auth_request con 0x7f4ad40c99a0 auth_method 0 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): get_auth_request con 0x7f4ad406ff20 auth_method 0 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): get_auth_request con 0x7f4ad415a5e0 auth_method 0 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ad9ea8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.426+0000 7f4adaeaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 10 monclient(hunting): handle_auth_done global_id 4402 payload 824 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 -- 10.20.193.18:0/1458983517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ad415a5e0 msgr2=0x7f4ad414d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ad415a5e0 0x7f4ad414d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 -- 10.20.193.18:0/1458983517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 msgr2=0x7f4ad40bf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 0x7f4ad40bf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.427+0000 7f4ada6a9640 1 -- 10.20.193.18:0/1458983517 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad40bdfd0 con 0x7f4ad406ff20 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4ada6a9640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad41f2f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006fd0 tx=0x7f4acc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 10 client.?.objecter ms_handle_connect 0x7f4ad406ff20 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc03c040 con 0x7f4ad406ff20 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.429+0000 7f4abeffd640 10 monclient: dump: 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 5 monclient: authenticate success, global_id 4402 2026-02-24T18:35:33.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _renew_subs 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad40be2b0 con 0x7f4ad406ff20 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 client.4402.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _renew_subs 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad40be870 con 0x7f4ad406ff20 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 1 librados: init done 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad4150860 con 0x7f4ad406ff20 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc02f9e0 con 0x7f4ad406ff20 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc042830 con 0x7f4ad406ff20 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4ad8ea6640 4 set_mon_vals no callback set 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.430+0000 7f4abeffd640 10 monclient: dump: 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.434 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.435 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.435 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.435 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.533+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4acc0429d0 con 0x7f4ad406ff20 2026-02-24T18:35:33.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.533+0000 7f4abeffd640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 0x7f4aa407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.533+0000 7f4ad9ea8640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 0x7f4aa407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.533+0000 7f4ad9ea8640 10 monclient: get_auth_request con 0x7f4aa4078e00 auth_method 0 2026-02-24T18:35:33.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.534+0000 7f4abeffd640 10 client.4402.objecter ms_handle_connect 0x7f4aa4078e00 2026-02-24T18:35:33.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.534+0000 7f4ad9ea8640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 0x7f4aa407b2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40c4400 tx=0x7f4ac0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.536+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f4acc042cf0 con 0x7f4ad406ff20 2026-02-24T18:35:33.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.536+0000 7f4abeffd640 10 client.4402.objecter ms_dispatch2 0x7f4ad4000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.536+0000 7f4abeffd640 3 client.4402.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.536+0000 7f4abeffd640 3 client.4402.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.536+0000 7f4abeffd640 20 client.4402.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.542+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4acc0876f0 con 0x7f4ad406ff20 2026-02-24T18:35:33.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.542+0000 7f4abeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.542+0000 7f4abeffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:33.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.625+0000 7f4adc934640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.625+0000 7f4adc934640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.625+0000 7f4adc934640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.625+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ad41582f0 con 0x7f4ad406ff20 2026-02-24T18:35:33.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.648+0000 7f4abeffd640 1 -- 10.20.193.18:0/1458983517 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+24707 (secure 0 0 0) 0x7f4acc08b9f0 con 0x7f4ad406ff20 2026-02-24T18:35:33.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.648+0000 7f4abeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:35:33.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.648+0000 7f4abeffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:33.650 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:33.650 INFO:teuthology.orchestra.run.trial018.stdout:{"epoch":28,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:35:32.309762+0000","last_up_change":"2026-02-24T18:35:26.937667+0000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:35:26.061331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:35:29.058214+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6817","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6819","nonce":1076077393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6821","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6823","nonce":1076077393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6829","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6831","nonce":1076077393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6825","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6827","nonce":1076077393}]},"public_addr":"10.20.193.18:6819/1076077393","cluster_addr":"10.20.193.18:6823/1076077393","heartbeat_back_addr":"10.20.193.18:6831/1076077393","heartbeat_front_addr":"10.20.193.18:6827/1076077393","state":["exists","up"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6800","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6801","nonce":2795974188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6802","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6803","nonce":2795974188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6806","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6807","nonce":2795974188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6804","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6805","nonce":2795974188}]},"public_addr":"10.20.193.32:6801/2795974188","cluster_addr":"10.20.193.32:6803/2795974188","heartbeat_back_addr":"10.20.193.32:6807/2795974188","heartbeat_front_addr":"10.20.193.32:6805/2795974188","state":["exists","up"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6825","nonce":3650974883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6826","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6827","nonce":3650974883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6830","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6831","nonce":3650974883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6829","nonce":3650974883}]},"public_addr":"10.20.193.110:6825/3650974883","cluster_addr":"10.20.193.110:6827/3650974883","heartbeat_back_addr":"10.20.193.110:6831/3650974883","heartbeat_front_addr":"10.20.193.110:6829/3650974883","state":["exists","up"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6826","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6827","nonce":230320724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6828","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6829","nonce":230320724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6832","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6833","nonce":230320724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6830","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6831","nonce":230320724}]},"public_addr":"10.20.193.112:6827/230320724","cluster_addr":"10.20.193.112:6829/230320724","heartbeat_back_addr":"10.20.193.112:6833/230320724","heartbeat_front_addr":"10.20.193.112:6831/230320724","state":["exists","up"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6800","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6801","nonce":3481454300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6802","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6803","nonce":3481454300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6806","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6807","nonce":3481454300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6804","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6805","nonce":3481454300}]},"public_addr":"10.20.193.18:6801/3481454300","cluster_addr":"10.20.193.18:6803/3481454300","heartbeat_back_addr":"10.20.193.18:6807/3481454300","heartbeat_front_addr":"10.20.193.18:6805/3481454300","state":["exists","up"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6808","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6809","nonce":2128834550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6810","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6811","nonce":2128834550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6814","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6815","nonce":2128834550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6812","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6813","nonce":2128834550}]},"public_addr":"10.20.193.32:6809/2128834550","cluster_addr":"10.20.193.32:6811/2128834550","heartbeat_back_addr":"10.20.193.32:6815/2128834550","heartbeat_front_addr":"10.20.193.32:6813/2128834550","state":["exists","up"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6801","nonce":3466222224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6803","nonce":3466222224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6807","nonce":3466222224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6805","nonce":3466222224}]},"public_addr":"10.20.193.110:6801/3466222224","cluster_addr":"10.20.193.110:6803/3466222224","heartbeat_back_addr":"10.20.193.110:6807/3466222224","heartbeat_front_addr":"10.20.193.110:6805/3466222224","state":["exists","up"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6810","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6811","nonce":92089351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6814","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6815","nonce":92089351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6821","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6823","nonce":92089351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6817","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6819","nonce":92089351}]},"public_addr":"10.20.193.112:6811/92089351","cluster_addr":"10.20.193.112:6815/92089351","heartbeat_back_addr":"10.20.193.112:6823/92089351","heartbeat_front_addr":"10.20.193.112:6819/92089351","state":["exists","up"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6808","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6809","nonce":1977979333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6810","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6811","nonce":1977979333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6814","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6815","nonce":1977979333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6812","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6813","nonce":1977979333}]},"public_addr":"10.20.193.18:6809/1977979333","cluster_addr":"10.20.193.18:6811/1977979333","heartbeat_back_addr":"10.20.193.18:6815/1977979333","heartbeat_front_addr":"10.20.193.18:6813/1977979333","state":["exists","up"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6816","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6817","nonce":1806399530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6818","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6819","nonce":1806399530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6822","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6823","nonce":1806399530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6820","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6821","nonce":1806399530}]},"public_addr":"10.20.193.32:6817/1806399530","cluster_addr":"10.20.193.32:6819/1806399530","heartbeat_back_addr":"10.20.193.32:6823/1806399530","heartbeat_front_addr":"10.20.193.32:6821/1806399530","state":["exists","up"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6809","nonce":4149932150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6811","nonce":4149932150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6815","nonce":4149932150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6813","nonce":4149932150}]},"public_addr":"10.20.193.110:6809/4149932150","cluster_addr":"10.20.193.110:6811/4149932150","heartbeat_back_addr":"10.20.193.110:6815/4149932150","heartbeat_front_addr":"10.20.193.110:6813/4149932150","state":["exists","up"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6812","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6813","nonce":2071089960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6816","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6818","nonce":2071089960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6824","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6825","nonce":2071089960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6820","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6822","nonce":2071089960}]},"public_addr":"10.20.193.112:6813/2071089960","cluster_addr":"10.20.193.112:6818/2071089960","heartbeat_back_addr":"10.20.193.112:6825/2071089960","heartbeat_front_addr":"10.20.193.112:6822/2071089960","state":["exists","up"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6816","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6818","nonce":392025864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6820","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6822","nonce":392025864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6828","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6830","nonce":392025864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6824","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6826","nonce":392025864}]},"public_addr":"10.20.193.18:6818/392025864","cluster_addr":"10.20.193.18:6822/392025864","heartbeat_back_addr":"10.20.193.18:6830/392025864","heartbeat_front_addr":"10.20.193.18:6826/392025864","state":["exists","up"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6824","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6825","nonce":316721302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6826","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6827","nonce":316721302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6830","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6831","nonce":316721302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6828","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6829","nonce":316721302}]},"public_addr":"10.20.193.32:6825/316721302","cluster_addr":"10.20.193.32:6827/316721302","heartbeat_back_addr":"10.20.193.32:6831/316721302","heartbeat_front_addr":"10.20.193.32:6829/316721302","state":["exists","up"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6817","nonce":3577843273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6818","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6819","nonce":3577843273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6822","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6823","nonce":3577843273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6821","nonce":3577843273}]},"public_addr":"10.20.193.110:6817/3577843273","cluster_addr":"10.20.193.110:6819/3577843273","heartbeat_back_addr":"10.20.193.110:6823/3577843273","heartbeat_front_addr":"10.20.193.110:6821/3577843273","state":["exists","up"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6802","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6803","nonce":4220209204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6804","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6805","nonce":4220209204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6808","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6809","nonce":4220209204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6806","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6807","nonce":4220209204}]},"public_addr":"10.20.193.112:6803/4220209204","cluster_addr":"10.20.193.112:6805/4220209204","heartbeat_back_addr":"10.20.193.112:6809/4220209204","heartbeat_front_addr":"10.20.193.112:6807/4220209204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.022230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.519970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.248082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.777136+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.483378+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.749510+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.414866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.123257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.766447+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.874254+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.504376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.120504+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.013826+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.378335+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.904134+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.610284+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:35:33.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 10 librados: watch_flush enter 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 10 librados: watch_flush exit 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 20 client.4402.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 10 client.4402.objecter successfully canceled tick 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 msgr2=0x7f4aa407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 0x7f4aa407b2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40c4400 tx=0x7f4ac0009290 comp rx=0 tx=0).stop 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 10 monclient: shutdown 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 msgr2=0x7f4ad41f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad41f2f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006fd0 tx=0x7f4acc0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 shutdown_connections 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4aa4078e00 0x7f4aa407b2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ad415a5e0 0x7f4ad414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4ad40c99a0 0x7f4ad40bf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 --2- 10.20.193.18:0/1458983517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ad406ff20 0x7f4ad41f2f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 >> 10.20.193.18:0/1458983517 conn(0x7f4ad40cd780 msgr2=0x7f4ad406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 shutdown_connections 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 -- 10.20.193.18:0/1458983517 wait complete. 2026-02-24T18:35:33.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.650+0000 7f4adc934640 1 librados: shutdown 2026-02-24T18:35:33.661 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-24T18:35:33.662 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-24T18:35:33.662 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-24T18:35:33.662 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-24T18:35:33.662 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-24T18:35:33.663 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-24T18:35:33.663 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-24T18:35:33.663 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-24T18:35:33.663 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-24T18:35:33.664 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-24T18:35:33.664 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2026-02-24T18:35:33.664 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2026-02-24T18:35:33.664 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2026-02-24T18:35:33.664 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2026-02-24T18:35:33.665 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2026-02-24T18:35:33.665 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 10 monclient: monmap: 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.746217+0000 2026-02-24T18:35:33.747 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.746217+0000 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.748 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 1 Processor -- start 2026-02-24T18:35:33.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 1 -- start start 2026-02-24T18:35:33.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.745+0000 7faa29196640 10 monclient: init 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient: _add_conns ranks=[5,4,2,0,1,3] 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7faa2406ff20 0x7faa24070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): picked mon.noname-c con 0x7faa2406ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7faa240c99a0 0x7faa24159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): picked mon.noname-b con 0x7faa240c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faa2415a410 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2415a860 con 0x7faa2415a410 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7faa24159a40 con 0x7faa240c99a0 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7faa240c7e90 con 0x7faa2406ff20 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.746+0000 7faa29196640 10 monclient(hunting): authenticate will time out at 1477.427897s 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7faa240c99a0 0x7faa24159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa23577640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7faa2415a410 0x7faa2414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58238/0 (socket says 10.20.193.18:58238) 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa23577640 1 -- 10.20.193.18:0/2944430671 learned_addr learned my addr 10.20.193.18:0/2944430671 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): get_auth_request con 0x7faa240c99a0 auth_method 0 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22d76640 1 --2- 10.20.193.18:0/2944430671 >> v2:10.20.193.110:3300/0 conn(0x7faa2406ff20 0x7faa24070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.747+0000 7faa22575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.748+0000 7faa22575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.748+0000 7faa22d76640 10 monclient(hunting): get_auth_request con 0x7faa2406ff20 auth_method 0 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.748+0000 7faa22d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.748+0000 7faa22d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.748+0000 7faa22d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 10 monclient(hunting): handle_auth_done global_id 4405 payload 274 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 1 -- 10.20.193.18:0/2944430671 >> v2:10.20.193.110:3300/0 conn(0x7faa2406ff20 msgr2=0x7faa24070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 1 --2- 10.20.193.18:0/2944430671 >> v2:10.20.193.110:3300/0 conn(0x7faa2406ff20 0x7faa24070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 1 -- 10.20.193.18:0/2944430671 >> v1:10.20.193.110:6789/0 conn(0x7faa2415a410 legacy=0x7faa2414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7faa22575640 1 -- 10.20.193.18:0/2944430671 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240c8170 con 0x7faa240c99a0 2026-02-24T18:35:33.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7fd11a4ea640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7fd11a4ea640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.749+0000 7fd11a4ea640 10 monclient: monmap: 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.750693+0000 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.750693+0000 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.757 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.758 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.758 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.758 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.758 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.758 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.750+0000 7faa22575640 1 --2- 10.20.193.18:0/2944430671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa240c99a0 0x7faa24159ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098a0 tx=0x7faa0c02ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d351e7f47f62936d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fd11a4ea640 1 Processor -- start 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.750+0000 7fa4898c9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.750+0000 7fa4898c9640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fa4898c9640 10 monclient: monmap: 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.751875+0000 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.751875+0000 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.759 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7ff332cf1640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7ff332cf1640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7ff332cf1640 10 monclient: monmap: 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.752061+0000 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.752061+0000 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.760 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fd11a4ea640 1 -- start start 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fd11a4ea640 10 monclient: init 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fa4898c9640 1 Processor -- start 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fa4898c9640 1 -- start start 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7fa4898c9640 10 monclient: init 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 1 -- 10.20.193.18:0/2944430671 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c03c070 con 0x7faa240c99a0 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: dump: 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa29196640 5 monclient: authenticate success, global_id 4405 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa29196640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 1 -- 10.20.193.18:0/2944430671 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0c02fae0 con 0x7faa240c99a0 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 1 -- 10.20.193.18:0/2944430671 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c037440 con 0x7faa240c99a0 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa21d74640 10 monclient: dump: 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa29196640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.751+0000 7faa29196640 4 set_mon_vals no callback set 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient: _add_conns ranks=[5,4,3,2,0,1] 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa4841527f0 0x7fa484073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa4841527f0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa48415d2c0 0x7fa48415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-b con 0x7fa48415d2c0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa48415d730 0x7fa48415f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-a con 0x7fa48415d730 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa484074b10 con 0x7fa48415d730 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa484073c20 con 0x7fa48415d2c0 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa484073f00 con 0x7fa4841527f0 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient: _add_conns ranks=[2,5,3,4,1,0] 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd11415f650 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd11415fa30 0x7fd1140d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4898c9640 10 monclient(hunting): authenticate will time out at 1477.433335s 2026-02-24T18:35:33.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7ff332cf1640 1 Processor -- start 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-c con 0x7fd11415fa30 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd1140d1ea0 0x7fd1140c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-a con 0x7fd1140d1ea0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1140d2450 con 0x7fd11415f650 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd1140c8dd0 con 0x7fd1140d1ea0 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd1140d2730 con 0x7fd11415fa30 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd11a4ea640 10 monclient(hunting): authenticate will time out at 1477.433441s 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7ff332cf1640 1 -- start start 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7ff332cf1640 10 monclient: init 2026-02-24T18:35:33.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 10 monclient: shutdown 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 -- 10.20.193.18:0/2944430671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa240c99a0 msgr2=0x7faa24159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 --2- 10.20.193.18:0/2944430671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa240c99a0 0x7faa24159ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098a0 tx=0x7faa0c02ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 -- 10.20.193.18:0/2944430671 shutdown_connections 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 --2- 10.20.193.18:0/2944430671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa240c99a0 0x7faa24159ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 --2- 10.20.193.18:0/2944430671 >> v2:10.20.193.110:3300/0 conn(0x7faa2406ff20 0x7faa24070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7faa29196640 1 -- 10.20.193.18:0/2944430671 >> 10.20.193.18:0/2944430671 conn(0x7faa240cd780 msgr2=0x7faa240cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fa4827fc640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa48415d2c0 0x7fa48415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7fd118a60640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd1140d1ea0 0x7fd1140c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7ff332cf1640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 1 -- 10.20.193.18:0/2944430671 shutdown_connections 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 1 -- 10.20.193.18:0/2944430671 wait complete. 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4837fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa48415d730 0x7fa48415f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa48415d2c0 0x7fa48415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52962/0 (socket says 10.20.193.18:52962) 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 1 -- 10.20.193.18:0/318165758 learned_addr learned my addr 10.20.193.18:0/318165758 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): get_auth_request con 0x7fa48415d2c0 auth_method 0 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.752+0000 7ff332cf1640 10 monclient: _add_conns ranks=[2,4,3,5,1,0] 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff32c15a310 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff32c15a6f0 0x7ff32c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-b con 0x7ff32c15a6f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff32c06e0b0 0x7ff32c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-a con 0x7ff32c06e0b0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c0cb7a0 con 0x7ff32c15a310 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0cba80 con 0x7ff32c06e0b0 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0cbd60 con 0x7ff32c15a6f0 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7ff332cf1640 10 monclient(hunting): authenticate will time out at 1477.434054s 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd1140d1ea0 0x7fd1140c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41136/0 (socket says 10.20.193.18:41136) 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 1 -- 10.20.193.18:0/3021168226 learned_addr learned my addr 10.20.193.18:0/3021168226 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): get_auth_request con 0x7fd1140d1ea0 auth_method 0 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4837fe640 10 monclient(hunting): get_auth_request con 0x7fa48415d730 auth_method 0 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fa4827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 10 monclient: monmap: 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 1 librados: starting msgr at 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 1 librados: starting objecter 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7faa29196640 1 Processor -- start 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.753+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 -- start start 2026-02-24T18:35:33.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 librados: setting wanted keys 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 librados: calling monclient init 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 10 monclient(hunting): handle_auth_done global_id 4408 payload 274 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 -- 10.20.193.18:0/318165758 >> v2:10.20.193.110:3300/0 conn(0x7fa4841527f0 msgr2=0x7fa484073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 --2- 10.20.193.18:0/318165758 >> v2:10.20.193.110:3300/0 conn(0x7fa4841527f0 0x7fa484073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 -- 10.20.193.18:0/318165758 >> v2:10.20.193.18:3300/0 conn(0x7fa48415d730 msgr2=0x7fa48415f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 --2- 10.20.193.18:0/318165758 >> v2:10.20.193.18:3300/0 conn(0x7fa48415d730 0x7fa48415f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7fa4827fc640 1 -- 10.20.193.18:0/318165758 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840741e0 con 0x7fa48415d2c0 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient: init 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 0x7faa241f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): picked mon.noname-b con 0x7faa2406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa240c99a0 0x7faa24150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): picked mon.noname-c con 0x7faa240c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa2415a410 0x7faa240c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): picked mon.noname-a con 0x7faa2415a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff32c15a6f0 0x7ff32c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7faa240c8a10 con 0x7faa2415a410 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa240c8730 con 0x7faa2406ff20 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa24150520 con 0x7faa240c99a0 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.754+0000 7faa29196640 10 monclient(hunting): authenticate will time out at 1477.435864s 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff32c15a6f0 0x7ff32c0ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52976/0 (socket says 10.20.193.18:52976) 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 1 -- 10.20.193.18:0/3473231187 learned_addr learned my addr 10.20.193.18:0/3473231187 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 10 monclient(hunting): get_auth_request con 0x7ff32c15a6f0 auth_method 0 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7ff323fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7faa22d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 0x7faa241f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 10 monclient(hunting): handle_auth_done global_id 4413 payload 274 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 1 -- 10.20.193.18:0/3021168226 >> v2:10.20.193.110:3300/0 conn(0x7fd11415fa30 msgr2=0x7fd1140d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 1 --2- 10.20.193.18:0/3021168226 >> v2:10.20.193.110:3300/0 conn(0x7fd11415fa30 0x7fd1140d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 1 -- 10.20.193.18:0/3021168226 >> v1:10.20.193.110:6789/0 conn(0x7fd11415f650 legacy=0x7fd1140ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7fd118a60640 1 -- 10.20.193.18:0/3021168226 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1140c9020 con 0x7fd1140d1ea0 2026-02-24T18:35:33.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7f0d93b03640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7f0d93b03640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.755+0000 7f0d93b03640 10 monclient: monmap: 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.756639+0000 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.756639+0000 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ff323fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ff323fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ff323fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7f0d93b03640 1 Processor -- start 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 0x7faa241f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52980/0 (socket says 10.20.193.18:52980) 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 1 -- 10.20.193.18:0/2857618340 learned_addr learned my addr 10.20.193.18:0/2857618340 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 10 monclient(hunting): get_auth_request con 0x7faa2406ff20 auth_method 0 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7faa22d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ffbd9334640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ffbd9334640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ffbd9334640 10 monclient: monmap: 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.757286+0000 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.757286+0000 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7f0d93b03640 1 -- start start 2026-02-24T18:35:33.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7f0d93b03640 10 monclient: init 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7fa4827fc640 1 --2- 10.20.193.18:0/318165758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 0x7fa48415f3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa48408f510 tx=0x7fa46c02ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc163e5a405f4b6 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7fc983f98640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7fc983f98640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7fc983f98640 10 monclient: monmap: 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.757700+0000 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.757700+0000 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.778 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.756+0000 7ffbd9334640 1 Processor -- start 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 1 -- start start 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient: init 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient: _add_conns ranks=[5,1,3,0,2,4] 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0d8c1527f0 0x7f0d8c073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-c con 0x7f0d8c1527f0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0d8c15d2c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0d8c15d730 0x7f0d8c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-a con 0x7f0d8c15d730 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8c073c20 con 0x7f0d8c15d2c0 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c074b10 con 0x7f0d8c15d730 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c073f00 con 0x7f0d8c1527f0 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7f0d93b03640 10 monclient(hunting): authenticate will time out at 1477.438223s 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 10 monclient(hunting): handle_auth_done global_id 4411 payload 274 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 1 -- 10.20.193.18:0/3473231187 >> v2:10.20.193.18:3300/0 conn(0x7ff32c06e0b0 msgr2=0x7ff32c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 1 --2- 10.20.193.18:0/3473231187 >> v2:10.20.193.18:3300/0 conn(0x7ff32c06e0b0 0x7ff32c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 1 -- 10.20.193.18:0/3473231187 >> v1:10.20.193.110:6789/0 conn(0x7ff32c15a310 legacy=0x7ff32c070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ff323fff640 1 -- 10.20.193.18:0/3473231187 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c0cc040 con 0x7ff32c15a6f0 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient: _add_conns ranks=[0,4,2,3,1,5] 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffbd40cb0a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ffbd406f510 0x7ffbd415d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-b con 0x7ffbd406f510 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ffbd40cccb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7faa22d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7fc983f98640 1 Processor -- start 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7fc983f98640 1 -- start start 2026-02-24T18:35:33.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7fc983f98640 10 monclient: init 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.757+0000 7ffbd9334640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd406f990 con 0x7ffbd40cb0a0 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7ffbd9334640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd406fc70 con 0x7ffbd40cccb0 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7ffbd9334640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7ffbd9334640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd406ff50 con 0x7ffbd406f510 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7ffbd9334640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7ffbd9334640 10 monclient(hunting): authenticate will time out at 1477.439015s 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient: _add_conns ranks=[3,0,1,5,2,4] 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 10 monclient: monmap: 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.759094+0000 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.759094+0000 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.782 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc97c15a0f0 0x7fc97c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-a con 0x7fc97c15a0f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc97c15a4d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc97c160350 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c160f60 con 0x7fc97c15a4d0 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c161240 con 0x7fc97c160350 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc97c0d2190 con 0x7fc97c15a0f0 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fc983f98640 10 monclient(hunting): authenticate will time out at 1477.439329s 2026-02-24T18:35:33.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 1 Processor -- start 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fa3c3bb3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fa3c3bb3640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fa3c3bb3640 10 monclient: monmap: 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.759707+0000 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.759707+0000 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 1 -- start start 2026-02-24T18:35:33.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7fecd79f7640 10 monclient: init 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f7fdede3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f7fdede3640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f0d92079640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0d8c15d730 0x7f0d8c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f7fdede3640 10 monclient: monmap: 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.759816+0000 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.759816+0000 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.785 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f5b0ef55640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f5b0ef55640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.758+0000 7f5b0ef55640 10 monclient: monmap: 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.759827+0000 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.759827+0000 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0d8c15d730 0x7f0d8c0ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41176/0 (socket says 10.20.193.18:41176) 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 1 -- 10.20.193.18:0/2558755144 learned_addr learned my addr 10.20.193.18:0/2558755144 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): get_auth_request con 0x7f0d8c15d730 auth_method 0 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 1 Processor -- start 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 1 -- start start 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 10 monclient: init 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f5b0ef55640 1 Processor -- start 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fd118a60640 1 --2- 10.20.193.18:0/3021168226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1140d1ea0 0x7fd1140c9b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce0a0 tx=0x7fd10802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08450b23b8d2bb4 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f5b0ef55640 1 -- start start 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f5b0ef55640 10 monclient: init 2026-02-24T18:35:33.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f7fdede3640 1 Processor -- start 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient: _add_conns ranks=[4,1,0,3,2,5] 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fecd0155eb0 0x7fecd015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-b con 0x7fecd0155eb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fecd0070210 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f7fdede3640 1 -- start start 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f7fdede3640 10 monclient: init 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecd015c430 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd015c960 con 0x7fecd015c430 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd006f450 con 0x7fecd0070210 2026-02-24T18:35:33.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fecd015cc40 con 0x7fecd0155eb0 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fecd79f7640 10 monclient(hunting): authenticate will time out at 1477.440723s 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7f0d92079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 10 monclient: _add_conns ranks=[3,5,4,2,1,0] 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 10 monclient(hunting): handle_auth_done global_id 4414 payload 824 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa3bc1527f0 0x7fa3bc073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 -- 10.20.193.18:0/2857618340 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa240c99a0 msgr2=0x7faa24150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa240c99a0 0x7faa24150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 -- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa2415a410 msgr2=0x7faa240c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa2415a410 0x7faa240c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 10 monclient: monmap: 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.760959+0000 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.760959+0000 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7faa22d76640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa241f9800 con 0x7faa2406ff20 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.759+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-a con 0x7fa3bc1527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-c con 0x7fa3bc15d2c0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa3bc15d730 0x7fa3bc15f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-b con 0x7fa3bc15d730 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa3bc074b10 con 0x7fa3bc1527f0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa3bc073c20 con 0x7fa3bc15d730 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa3bc073f00 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa3c3bb3640 10 monclient(hunting): authenticate will time out at 1477.441163s 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient: _add_conns ranks=[2,1,5,3,0,4] 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b081527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5b08073530 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 1 Processor -- start 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5b0815fad0 0x7f5b080ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-c con 0x7f5b0815fad0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b080739a0 con 0x7f5b08073530 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0815d9d0 con 0x7f5b081527f0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5b08074b10 con 0x7f5b0815fad0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f5b0ef55640 10 monclient(hunting): authenticate will time out at 1477.441125s 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 1 -- start start 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7fa0d3c2b640 10 monclient: init 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient: _add_conns ranks=[2,0,1,4,3,5] 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7fd8153bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7fd8153f90 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7fd80cc700 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd8159d80 con 0x7f7fd8153f90 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd815a060 con 0x7f7fd80cc700 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd815a340 con 0x7f7fd8153bb0 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f7fdede3640 10 monclient(hunting): authenticate will time out at 1477.441750s 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 10 monclient(hunting): handle_auth_done global_id 4419 payload 274 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 1 -- 10.20.193.18:0/2558755144 >> v2:10.20.193.110:3300/0 conn(0x7f0d8c1527f0 msgr2=0x7f0d8c073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7ff323fff640 1 --2- 10.20.193.18:0/3473231187 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c15a6f0 0x7ff32c0ccb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0727e0 tx=0x7ff31402ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e6ce28f039fec0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 1 --2- 10.20.193.18:0/2558755144 >> v2:10.20.193.110:3300/0 conn(0x7f0d8c1527f0 0x7f0d8c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 1 -- 10.20.193.18:0/2558755144 >> v1:10.20.193.32:6789/0 conn(0x7f0d8c15d2c0 legacy=0x7f0d8c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.760+0000 7f0d92079640 1 -- 10.20.193.18:0/2558755144 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0741e0 con 0x7f0d8c15d730 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient: _add_conns ranks=[3,4,0,2,1,5] 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa0cc1527f0 0x7fa0cc073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-a con 0x7fa0cc1527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa0cc15d2c0 0x7fa0cc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-b con 0x7fa0cc15d2c0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa0cc15d730 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0cc073c20 con 0x7fa0cc15d730 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa0cc074b10 con 0x7fa0cc1527f0 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa0cc073f00 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.761+0000 7fa0d3c2b640 10 monclient(hunting): authenticate will time out at 1477.442454s 2026-02-24T18:35:33.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f0d92079640 1 --2- 10.20.193.18:0/2558755144 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c15d730 0x7f0d8c0ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c06e9a0 tx=0x7f0d8802f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e95fa3036629b8d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 10 monclient: monmap: 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.763096+0000 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.763096+0000 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 1 Processor -- start 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 1 -- start start 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7f4631e88640 10 monclient: init 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7fc98150c640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fc97c15a4d0 0x7fc97c0c4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58522/0 (socket says 10.20.193.18:58522) 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.762+0000 7fc98150c640 1 -- 10.20.193.18:0/3812608940 learned_addr learned my addr 10.20.193.18:0/3812608940 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fc981d0d640 1 --2- 10.20.193.18:0/3812608940 >> v2:10.20.193.18:3300/0 conn(0x7fc97c15a0f0 0x7fc97c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient: _add_conns ranks=[0,4,2,3,1,5] 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f462c1527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f462c073530 0x7f462c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-b con 0x7f462c073530 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f462c15f900 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462c0739a0 con 0x7f462c1527f0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462c15fd90 con 0x7f462c15f900 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f462c074b10 con 0x7f462c073530 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f4631e88640 10 monclient(hunting): authenticate will time out at 1477.444504s 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fc981d0d640 10 monclient(hunting): get_auth_request con 0x7fc97c15a0f0 auth_method 0 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fc981d0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fc981d0d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fc981d0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7fa3c1127640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f66f16f5640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f66f16f5640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f66f16f5640 10 monclient: monmap: 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.764491+0000 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.764491+0000 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5b0815fad0 0x7f5b080ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5b0815fad0 0x7f5b080ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44040/0 (socket says 10.20.193.18:44040) 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/334417414 learned_addr learned my addr 10.20.193.18:0/334417414 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 10 monclient(hunting): get_auth_request con 0x7f5b0815fad0 auth_method 0 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.763+0000 7f5b0d4cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44030/0 (socket says 10.20.193.18:44030) 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 1 -- 10.20.193.18:0/2573679146 learned_addr learned my addr 10.20.193.18:0/2573679146 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 10 monclient(hunting): get_auth_request con 0x7fa3bc15d2c0 auth_method 0 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa3c1127640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa0d21a1640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa0cc15d730 0x7fa0cc0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58548/0 (socket says 10.20.193.18:58548) 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fa0d21a1640 1 -- 10.20.193.18:0/1722328463 learned_addr learned my addr 10.20.193.18:0/1722328463 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 Processor -- start 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 -- start start 2026-02-24T18:35:33.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient: init 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7ffbd2ffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7ffbd40cb0a0 0x7ffbd415ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58512/0 (socket says 10.20.193.18:58512) 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7ffbd2ffd640 1 -- 10.20.193.18:0/3036568195 learned_addr learned my addr 10.20.193.18:0/3036568195 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient: _add_conns ranks=[4,1,5,2,0,3] 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66ec1527f0 0x7f66ec073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-b con 0x7f66ec1527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f66ec15d2c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f66ec15d730 0x7f66ec0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-c con 0x7f66ec15d730 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ec073c20 con 0x7f66ec15d2c0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec074b10 con 0x7f66ec1527f0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec073f00 con 0x7f66ec15d730 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7f66f16f5640 10 monclient(hunting): authenticate will time out at 1477.445829s 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7faa22d76640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 0x7faa241f92c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa1800ebe0 tx=0x7faa1800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.764+0000 7faa037fe640 10 client.?.objecter ms_handle_connect 0x7faa2406ff20 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7faa037fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7fa3c1127640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7fa3c1127640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7fa3c1127640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_done global_id 4406 payload 274 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/334417414 >> v1:10.20.193.110:6789/0 conn(0x7f5b081527f0 legacy=0x7f5b0815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/334417414 >> v1:10.20.193.32:6789/0 conn(0x7f5b08073530 legacy=0x7f5b0815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.765+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/334417414 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08073c20 con 0x7f5b0815fad0 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 10 monclient(hunting): handle_auth_done global_id 4409 payload 274 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 -- 10.20.193.18:0/2573679146 >> v2:10.20.193.32:3300/0 conn(0x7fa3bc15d730 msgr2=0x7fa3bc15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 --2- 10.20.193.18:0/2573679146 >> v2:10.20.193.32:3300/0 conn(0x7fa3bc15d730 0x7fa3bc15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 -- 10.20.193.18:0/2573679146 >> v2:10.20.193.18:3300/0 conn(0x7fa3bc1527f0 msgr2=0x7fa3bc073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 --2- 10.20.193.18:0/2573679146 >> v2:10.20.193.18:3300/0 conn(0x7fa3bc1527f0 0x7fa3bc073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.766+0000 7fa3c1127640 1 -- 10.20.193.18:0/2573679146 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc0741e0 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7ffbd27fc640 1 --2- 10.20.193.18:0/3036568195 >> v2:10.20.193.32:3300/0 conn(0x7ffbd406f510 0x7ffbd415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7f0b8452c640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7f0b8452c640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7f0b8452c640 10 monclient: monmap: 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.768551+0000 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.768551+0000 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7ffbd27fc640 10 monclient(hunting): get_auth_request con 0x7ffbd406f510 auth_method 0 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7ffbd27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7ffbd27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.767+0000 7ffbd27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa0d119f640 1 --2- 10.20.193.18:0/1722328463 >> v2:10.20.193.32:3300/0 conn(0x7fa0cc15d2c0 0x7fa0cc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 Processor -- start 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 -- start start 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient: init 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa0d119f640 10 monclient(hunting): get_auth_request con 0x7fa0cc15d2c0 auth_method 0 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa0d119f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa0d119f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa0d119f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f3ed72f1640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f3ed72f1640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f3ed72f1640 10 monclient: monmap: 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.769518+0000 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.769518+0000 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.802 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient: _add_conns ranks=[3,4,5,0,2,1] 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0b7c15f650 0x7f0b7c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-a con 0x7f0b7c15f650 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0b7c0ceca0 0x7f0b7c0d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 1 -- 10.20.193.18:0/318165758 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c03c070 con 0x7fa48415d2c0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: dump: 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 1 -- 10.20.193.18:0/318165758 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c02fb50 con 0x7fa48415d2c0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 1 -- 10.20.193.18:0/318165758 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c037440 con 0x7fa48415d2c0 2026-02-24T18:35:33.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7fa481ffb640 10 monclient: dump: 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-b con 0x7f0b7c0ceca0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-c con 0x7f0b7c0d1ea0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0c8dd0 con 0x7f0b7c15f650 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0d2450 con 0x7f0b7c0ceca0 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0d2730 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.768+0000 7f0b8452c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 5 monclient: authenticate success, global_id 4408 2026-02-24T18:35:33.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 4 set_mon_vals no callback set 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0b8452c640 10 monclient(hunting): authenticate will time out at 1477.450014s 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fecd0155eb0 0x7fecd015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 Processor -- start 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 10 monclient(hunting): handle_auth_done global_id 4422 payload 274 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 1 -- 10.20.193.18:0/3812608940 >> v1:10.20.193.32:6789/0 conn(0x7fc97c160350 legacy=0x7fc97c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 1 -- 10.20.193.18:0/3812608940 >> v1:10.20.193.18:6789/0 conn(0x7fc97c15a4d0 legacy=0x7fc97c0c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66ec1527f0 0x7f66ec073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fc981d0d640 1 -- 10.20.193.18:0/3812608940 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c0d1860 con 0x7fc97c15a0f0 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa3c1127640 1 --2- 10.20.193.18:0/2573679146 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc06cdf0 tx=0x7fa3b002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a9cae703e936b9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 -- start start 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient: init 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa0d119f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa0d119f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa0d119f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 1 -- 10.20.193.18:0/2558755144 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8802fcd0 con 0x7f0d8c15d730 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 10 monclient: dump: 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d90876640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d93b03640 5 monclient: authenticate success, global_id 4419 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f0d93b03640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 10 monclient: shutdown 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 -- 10.20.193.18:0/318165758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 msgr2=0x7fa48415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 --2- 10.20.193.18:0/318165758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 0x7fa48415f3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa48408f510 tx=0x7fa46c02ee90 comp rx=0 tx=0).stop 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fecd0155eb0 0x7fecd015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52988/0 (socket says 10.20.193.18:52988) 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 1 -- 10.20.193.18:0/2004343576 learned_addr learned my addr 10.20.193.18:0/2004343576 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 10 monclient(hunting): get_auth_request con 0x7fecd0155eb0 auth_method 0 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fecd576c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f5b0d4cb640 1 --2- 10.20.193.18:0/334417414 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b080ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0402f2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38771537424845c3 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 -- 10.20.193.18:0/318165758 shutdown_connections 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 --2- 10.20.193.18:0/318165758 >> v2:10.20.193.18:3300/0 conn(0x7fa48415d730 0x7fa48415f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 --2- 10.20.193.18:0/318165758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 0x7fa48415f3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 --2- 10.20.193.18:0/318165758 >> v2:10.20.193.110:3300/0 conn(0x7fa4841527f0 0x7fa484073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 -- 10.20.193.18:0/318165758 >> 10.20.193.18:0/318165758 conn(0x7fa4840ce730 msgr2=0x7fa4840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 -- 10.20.193.18:0/318165758 shutdown_connections 2026-02-24T18:35:33.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7fa4898c9640 1 -- 10.20.193.18:0/318165758 wait complete. 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66ec1527f0 0x7f66ec073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53036/0 (socket says 10.20.193.18:53036) 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 1 -- 10.20.193.18:0/1260801641 learned_addr learned my addr 10.20.193.18:0/1260801641 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 10 monclient(hunting): get_auth_request con 0x7f66ec1527f0 auth_method 0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f66eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient: _add_conns ranks=[5,1,2,3,4,0] 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3ed01527f0 0x7f3ed0073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f462bfff640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f462c15f900 0x7f462c0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58276/0 (socket says 10.20.193.18:58276) 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f462bfff640 1 -- 10.20.193.18:0/1630791889 learned_addr learned my addr 10.20.193.18:0/1630791889 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-c con 0x7f3ed01527f0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3ed015d2c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ed015d730 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed0073c20 con 0x7f3ed015d2c0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed0073f00 con 0x7f3ed015d730 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed0074b10 con 0x7f3ed01527f0 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.769+0000 7f3ed72f1640 10 monclient(hunting): authenticate will time out at 1477.450992s 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 10 monclient(hunting): handle_auth_done global_id 4417 payload 274 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 1 -- 10.20.193.18:0/1722328463 >> v2:10.20.193.18:3300/0 conn(0x7fa0cc1527f0 msgr2=0x7fa0cc073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 1 --2- 10.20.193.18:0/1722328463 >> v2:10.20.193.18:3300/0 conn(0x7fa0cc1527f0 0x7fa0cc073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 10 monclient: monmap: 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 1 -- 10.20.193.18:0/1722328463 >> v1:10.20.193.18:6789/0 conn(0x7fa0cc15d730 legacy=0x7fa0cc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa0d119f640 1 -- 10.20.193.18:0/1722328463 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc0741e0 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 librados: starting msgr at 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 librados: starting objecter 2026-02-24T18:35:33.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 Processor -- start 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fecd576c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fecd576c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fecd576c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 -- start start 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 librados: setting wanted keys 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 1 librados: calling monclient init 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.770+0000 7fa4898c9640 10 monclient: init 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 0x7fa48415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa4841527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 0x7fa4841fa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-b con 0x7fa48415d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa48415d730 0x7fa4841fac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): picked mon.noname-a con 0x7fa48415d730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa484074fa0 con 0x7fa48415d730 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840747a0 con 0x7fa48415d2c0 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa484075120 con 0x7fa4841527f0 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7fa4898c9640 10 monclient(hunting): authenticate will time out at 1477.452158s 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7f7fdcb58640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f7fd8153bb0 0x7f7fd80cbbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58274/0 (socket says 10.20.193.18:58274) 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.771+0000 7f7fdcb58640 1 -- 10.20.193.18:0/2835151145 learned_addr learned my addr 10.20.193.18:0/2835151145 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 10 monclient(hunting): handle_auth_done global_id 4420 payload 274 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 1 -- 10.20.193.18:0/2004343576 >> v1:10.20.193.32:6789/0 conn(0x7fecd0070210 legacy=0x7fecd015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 1 -- 10.20.193.18:0/2004343576 >> v1:10.20.193.18:6789/0 conn(0x7fecd015c430 legacy=0x7fecd006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7fecd576c640 1 -- 10.20.193.18:0/2004343576 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd006f730 con 0x7fecd0155eb0 2026-02-24T18:35:33.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 1 -- 10.20.193.18:0/2558755144 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d880373f0 con 0x7f0d8c15d730 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 1 -- 10.20.193.18:0/2558755144 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880377c0 con 0x7f0d8c15d730 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d90876640 10 monclient: dump: 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d93b03640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.772+0000 7f0d93b03640 4 set_mon_vals no callback set 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 10 monclient: shutdown 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 -- 10.20.193.18:0/2558755144 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c15d730 msgr2=0x7f0d8c0ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 --2- 10.20.193.18:0/2558755144 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c15d730 0x7f0d8c0ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c06e9a0 tx=0x7f0d8802f270 comp rx=0 tx=0).stop 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 -- 10.20.193.18:0/2558755144 shutdown_connections 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 --2- 10.20.193.18:0/2558755144 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c15d730 0x7f0d8c0ccf70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 --2- 10.20.193.18:0/2558755144 >> v2:10.20.193.110:3300/0 conn(0x7f0d8c1527f0 0x7f0d8c073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 -- 10.20.193.18:0/2558755144 >> 10.20.193.18:0/2558755144 conn(0x7f0d8c0ce730 msgr2=0x7f0d8c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 -- 10.20.193.18:0/2558755144 shutdown_connections 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 1 -- 10.20.193.18:0/2558755144 wait complete. 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 10 monclient(hunting): handle_auth_done global_id 4423 payload 274 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 1 -- 10.20.193.18:0/1260801641 >> v2:10.20.193.110:3300/0 conn(0x7f66ec15d730 msgr2=0x7f66ec0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 1 --2- 10.20.193.18:0/1260801641 >> v2:10.20.193.110:3300/0 conn(0x7f66ec15d730 0x7f66ec0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 1 -- 10.20.193.18:0/1260801641 >> v1:10.20.193.32:6789/0 conn(0x7f66ec15d2c0 legacy=0x7f66ec15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f66eaffd640 1 -- 10.20.193.18:0/1260801641 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0741e0 con 0x7f66ec1527f0 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7fa0d119f640 1 --2- 10.20.193.18:0/1722328463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 0x7fa0cc15f3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc08f510 tx=0x7fa0c002ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5b5dd449705ed5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7fc981d0d640 1 --2- 10.20.193.18:0/3812608940 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1617d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098a0 tx=0x7fc96c02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be29c2d056c8a43 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.773+0000 7f0d93b03640 10 monclient: monmap: 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 librados: starting msgr at 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 librados: starting objecter 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 Processor -- start 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 -- start start 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 librados: setting wanted keys 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 librados: calling monclient init 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient: init 2026-02-24T18:35:33.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7fecd576c640 1 --2- 10.20.193.18:0/2004343576 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd015d1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009970 tx=0x7fecc002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2316fce0ea88dc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 0x7f0d8c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-a con 0x7f0d8c1527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d8c15d2c0 0x7f0d8c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-c con 0x7f0d8c15d2c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d8c15d730 0x7f0d8c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): picked mon.noname-b con 0x7f0d8c15d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c074fa0 con 0x7f0d8c1527f0 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0747a0 con 0x7f0d8c15d730 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c075120 con 0x7f0d8c15d2c0 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.774+0000 7f0d93b03640 10 monclient(hunting): authenticate will time out at 1477.455940s 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7f66eaffd640 1 --2- 10.20.193.18:0/1260801641 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec1527f0 0x7f66ec073530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06cdf0 tx=0x7f66e002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e00cdc71589f931 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 1 -- 10.20.193.18:0/3473231187 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31403c070 con 0x7ff32c15a6f0 2026-02-24T18:35:33.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: dump: 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 1 -- 10.20.193.18:0/3473231187 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31402fa90 con 0x7ff32c15a6f0 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 1 -- 10.20.193.18:0/3473231187 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31402fe60 con 0x7ff32c15a6f0 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 5 monclient: authenticate success, global_id 4411 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 4 set_mon_vals no callback set 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff3237fe640 10 monclient: dump: 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 10 monclient: shutdown 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 1 -- 10.20.193.18:0/3473231187 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c15a6f0 msgr2=0x7ff32c0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.775+0000 7ff332cf1640 1 --2- 10.20.193.18:0/3473231187 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c15a6f0 0x7ff32c0ccb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0727e0 tx=0x7ff31402ee60 comp rx=0 tx=0).stop 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 -- 10.20.193.18:0/3473231187 shutdown_connections 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 --2- 10.20.193.18:0/3473231187 >> v2:10.20.193.18:3300/0 conn(0x7ff32c06e0b0 0x7ff32c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 --2- 10.20.193.18:0/3473231187 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c15a6f0 0x7ff32c0ccb20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 -- 10.20.193.18:0/3473231187 >> 10.20.193.18:0/3473231187 conn(0x7ff32c0ca1f0 msgr2=0x7ff32c0ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 -- 10.20.193.18:0/3473231187 shutdown_connections 2026-02-24T18:35:33.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 -- 10.20.193.18:0/3473231187 wait complete. 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 1 -- 10.20.193.18:0/1630791889 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191392251 0 0) 0x7f4620002ca0 con 0x7f462c15f900 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 10 monclient(hunting): my global_id is 4412 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f462a7fc640 1 -- 10.20.193.18:0/1630791889 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4610003620 con 0x7f462c15f900 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 10 monclient: monmap: 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 1 -- 10.20.193.18:0/334417414 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0402fd00 con 0x7f5b0815fad0 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: dump: 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.816 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 1 -- 10.20.193.18:0/334417414 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b040373f0 con 0x7f5b0815fad0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 5 monclient: authenticate success, global_id 4406 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 1 -- 10.20.193.18:0/334417414 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040377c0 con 0x7f5b0815fad0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 4 set_mon_vals no callback set 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5aff7fe640 10 monclient: dump: 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f0b82aa2640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 1 -- 10.20.193.18:0/2573679146 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b003c070 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: dump: 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.817 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c3bb3640 5 monclient: authenticate success, global_id 4409 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 1 -- 10.20.193.18:0/2573679146 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b002fb50 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c3bb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c3bb3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c3bb3640 4 set_mon_vals no callback set 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 1 -- 10.20.193.18:0/2573679146 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0037440 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7fa3c0926640 10 monclient: dump: 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 librados: starting msgr at 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 librados: starting objecter 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 10 monclient: shutdown 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 -- 10.20.193.18:0/334417414 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 msgr2=0x7f5b080ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 --2- 10.20.193.18:0/334417414 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b080ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0402f2a0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 Processor -- start 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 -- 10.20.193.18:0/334417414 shutdown_connections 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 --2- 10.20.193.18:0/334417414 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b080ccf70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 -- 10.20.193.18:0/334417414 >> 10.20.193.18:0/334417414 conn(0x7f5b080ce730 msgr2=0x7f5b080ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 -- 10.20.193.18:0/334417414 shutdown_connections 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 -- start start 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 librados: setting wanted keys 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 1 librados: calling monclient init 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7ff332cf1640 10 monclient: init 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.776+0000 7f5b0ef55640 1 -- 10.20.193.18:0/334417414 wait complete. 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44048/0 (socket says 10.20.193.18:44048) 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 1 -- 10.20.193.18:0/3668237198 learned_addr learned my addr 10.20.193.18:0/3668237198 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): get_auth_request con 0x7f0b7c0d1ea0 auth_method 0 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 1 -- 10.20.193.18:0/3036568195 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817245059 0 0) 0x7ffbd406fc70 con 0x7ffbd40cccb0 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 10 monclient(hunting): my global_id is 4415 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ffbd1ffb640 1 -- 10.20.193.18:0/3036568195 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbc0003620 con 0x7ffbd40cccb0 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 10 monclient: shutdown 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f462affd640 1 --2- 10.20.193.18:0/1630791889 >> v2:10.20.193.32:3300/0 conn(0x7f462c073530 0x7f462c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/2573679146 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc15d2c0 msgr2=0x7fa3bc15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/2573679146 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc06cdf0 tx=0x7fa3b002ee90 comp rx=0 tx=0).stop 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/2573679146 shutdown_connections 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/2573679146 >> v2:10.20.193.32:3300/0 conn(0x7fa3bc15d730 0x7fa3bc15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/2573679146 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc15d2c0 0x7fa3bc15f3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/2573679146 >> v2:10.20.193.18:3300/0 conn(0x7fa3bc1527f0 0x7fa3bc073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/2573679146 >> 10.20.193.18:0/2573679146 conn(0x7fa3bc0ce730 msgr2=0x7fa3bc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/2573679146 shutdown_connections 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c06e0b0 0x7ff32c1f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-b con 0x7ff32c06e0b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/2573679146 wait complete. 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff32c15a310 0x7ff32c1fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-c con 0x7ff32c15a310 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 0x7ff32c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): picked mon.noname-a con 0x7ff32c1ee320 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c06e9f0 con 0x7ff32c1ee320 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0cc600 con 0x7ff32c06e0b0 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1616a0 con 0x7ff32c15a310 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7ff332cf1640 10 monclient(hunting): authenticate will time out at 1477.458439s 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 10 monclient: monmap: 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 0x7fa48415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f3ed5066640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3ed01527f0 0x7f3ed0073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f0b82aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 librados: starting msgr at 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 librados: starting objecter 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 Processor -- start 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 -- start start 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 librados: setting wanted keys 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 1 librados: calling monclient init 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7f5b0ef55640 10 monclient: init 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 0x7fa48415ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44072/0 (socket says 10.20.193.18:44072) 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 1 -- 10.20.193.18:0/3856407580 learned_addr learned my addr 10.20.193.18:0/3856407580 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 10 monclient(hunting): get_auth_request con 0x7fa4841527f0 auth_method 0 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.777+0000 7fa482ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient: monmap: 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.822 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3ed01527f0 0x7f3ed0073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44060/0 (socket says 10.20.193.18:44060) 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 1 -- 10.20.193.18:0/2208805029 learned_addr learned my addr 10.20.193.18:0/2208805029 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 10 monclient(hunting): get_auth_request con 0x7f3ed01527f0 auth_method 0 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f3ed5066640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 librados: starting msgr at 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 librados: starting objecter 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 Processor -- start 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 -- start start 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 librados: setting wanted keys 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 librados: calling monclient init 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient: init 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b08073530 0x7f5b081f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-a con 0x7f5b08073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b081527f0 0x7f5b08149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-b con 0x7f5b081527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b081f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): picked mon.noname-c con 0x7f5b0815fad0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080744c0 con 0x7f5b08073530 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080741e0 con 0x7f5b081527f0 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080747a0 con 0x7f5b0815fad0 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f5b0ef55640 10 monclient(hunting): authenticate will time out at 1477.459558s 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 10 monclient(hunting): handle_auth_done global_id 4421 payload 274 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 -- 10.20.193.18:0/3668237198 >> v2:10.20.193.32:3300/0 conn(0x7f0b7c0ceca0 msgr2=0x7f0b7c0d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 --2- 10.20.193.18:0/3668237198 >> v2:10.20.193.32:3300/0 conn(0x7f0b7c0ceca0 0x7f0b7c0d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 -- 10.20.193.18:0/3668237198 >> v2:10.20.193.18:3300/0 conn(0x7f0b7c15f650 msgr2=0x7f0b7c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 --2- 10.20.193.18:0/3668237198 >> v2:10.20.193.18:3300/0 conn(0x7f0b7c15f650 0x7f0b7c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 0x7fa3bc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-c con 0x7fa3bc1527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7f0b82aa2640 1 -- 10.20.193.18:0/3668237198 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0c9020 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa3bc15d2c0 0x7fa3bc1f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-a con 0x7fa3bc15d2c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa3bc15d730 0x7fa3bc1f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): picked mon.noname-b con 0x7fa3bc15d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc074fa0 con 0x7fa3bc15d2c0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0747a0 con 0x7fa3bc15d730 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc075120 con 0x7fa3bc1527f0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.778+0000 7fa3c3bb3640 10 monclient(hunting): authenticate will time out at 1477.459940s 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 1 -- 10.20.193.18:0/3021168226 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10802fc40 con 0x7fd1140d1ea0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: dump: 2026-02-24T18:35:33.825 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 5 monclient: authenticate success, global_id 4413 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 1 -- 10.20.193.18:0/3021168226 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10802fde0 con 0x7fd1140d1ea0 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 1 -- 10.20.193.18:0/3021168226 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108037610 con 0x7fd1140d1ea0 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 4 set_mon_vals no callback set 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd112ffd640 10 monclient: dump: 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f462affd640 10 monclient(hunting): get_auth_request con 0x7f462c073530 auth_method 0 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f462affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f462affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f462affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fa482ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fa482ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fa482ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f3ed5066640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f3ed5066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7f3ed5066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 10 monclient: shutdown 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3021168226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1140d1ea0 msgr2=0x7fd1140c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3021168226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1140d1ea0 0x7fd1140c9b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce0a0 tx=0x7fd10802f1e0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3021168226 shutdown_connections 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3021168226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1140d1ea0 0x7fd1140c9b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3021168226 >> v2:10.20.193.110:3300/0 conn(0x7fd11415fa30 0x7fd1140d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3021168226 >> 10.20.193.18:0/3021168226 conn(0x7fd114072fc0 msgr2=0x7fd1140733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3021168226 shutdown_connections 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.779+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3021168226 wait complete. 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient: monmap: 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.827 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 librados: starting msgr at 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 librados: starting objecter 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 Processor -- start 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 -- start start 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 librados: setting wanted keys 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 librados: calling monclient init 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient: init 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18017070 con 0x7faa2406ff20 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: dump: 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1800cc40 con 0x7faa2406ff20 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 5 monclient: authenticate success, global_id 4414 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18013470 con 0x7faa2406ff20 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa037fe640 10 monclient: dump: 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.828 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1140d1ea0 0x7fd1140ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-c con 0x7fd1140d1ea0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd11415f650 0x7fd1141f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-b con 0x7fd11415f650 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 0x7fd11414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa21573640 4 set_mon_vals no callback set 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 10 monclient: _renew_subs 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa241519e0 con 0x7faa2406ff20 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 10 client.4414.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 10 monclient: _renew_subs 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7fd11a4ea640 10 monclient(hunting): picked mon.noname-a con 0x7fd11415fa30 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd114160150 con 0x7fd11415fa30 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1140c95e0 con 0x7fd11415f650 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa24151fa0 con 0x7faa2406ff20 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.780+0000 7faa29196640 1 librados: init done 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141602d0 con 0x7fd1140d1ea0 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7fd11a4ea640 10 monclient(hunting): authenticate will time out at 1477.462023s 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 10 client.4414.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 15 client.4414.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 20 client.4414.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 10 client.4414.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa241508b0 con 0x7faa2406ff20 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7ffbd27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7ffbd27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.781+0000 7ffbd27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 10 monclient(hunting): handle_auth_done global_id 4424 payload 824 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 -- 10.20.193.18:0/3856407580 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 msgr2=0x7fa4841fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa48415d2c0 0x7fa4841fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 -- 10.20.193.18:0/3856407580 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa48415d730 msgr2=0x7fa4841fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa48415d730 0x7fa4841fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7fa482ffd640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48414f7c0 con 0x7fa4841527f0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 10 monclient(hunting): handle_auth_done global_id 4427 payload 274 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 1 -- 10.20.193.18:0/2208805029 >> v1:10.20.193.110:6789/0 conn(0x7f3ed015d730 legacy=0x7f3ed00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 1 -- 10.20.193.18:0/2208805029 >> v1:10.20.193.32:6789/0 conn(0x7f3ed015d2c0 legacy=0x7f3ed015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f3ed5066640 1 -- 10.20.193.18:0/2208805029 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed00741e0 con 0x7f3ed01527f0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7f0b82aa2640 1 --2- 10.20.193.18:0/3668237198 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ce0a0 tx=0x7f0b7802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aededb3a96b4650 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 10 monclient(hunting): handle_auth_done global_id 4426 payload 274 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 1 -- 10.20.193.18:0/3036568195 >> v1:10.20.193.110:6789/0 conn(0x7ffbd40cccb0 legacy=0x7ffbd40cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 1 -- 10.20.193.18:0/3036568195 >> v1:10.20.193.18:6789/0 conn(0x7ffbd40cb0a0 legacy=0x7ffbd415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.782+0000 7ffbd27fc640 1 -- 10.20.193.18:0/3036568195 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd4070230 con 0x7ffbd406f510 2026-02-24T18:35:33.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.784+0000 7fa482ffd640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 0x7fa48415ee30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa47800b780 tx=0x7fa47800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.784+0000 7fa45bfff640 10 client.?.objecter ms_handle_connect 0x7fa4841527f0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.784+0000 7fa45bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.784+0000 7f3ed5066640 1 --2- 10.20.193.18:0/2208805029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 0x7f3ed0073530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009970 tx=0x7f3ec002eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e22a8e7b2744e19 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 1 -- 10.20.193.18:0/3812608940 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c03c070 con 0x7fc97c15a0f0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: dump: 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 1 -- 10.20.193.18:0/3812608940 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96c02fad0 con 0x7fc97c15a0f0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 1 -- 10.20.193.18:0/3812608940 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c037440 con 0x7fc97c15a0f0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 5 monclient: authenticate success, global_id 4422 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 4 set_mon_vals no callback set 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc980d0b640 10 monclient: dump: 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.832 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 0x7ff32c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046117742 0 0) 0x7f7fd815a340 con 0x7f7fd8153bb0 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 10 monclient(hunting): my global_id is 4418 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fc4003620 con 0x7f7fd8153bb0 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 0x7ff32c158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41258/0 (socket says 10.20.193.18:41258) 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 1 -- 10.20.193.18:0/2144418042 learned_addr learned my addr 10.20.193.18:0/2144418042 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 10 monclient(hunting): get_auth_request con 0x7ff32c1ee320 auth_method 0 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7ff331267640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 10 monclient: shutdown 2026-02-24T18:35:33.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 -- 10.20.193.18:0/3812608940 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a0f0 msgr2=0x7fc97c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 --2- 10.20.193.18:0/3812608940 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1617d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098a0 tx=0x7fc96c02ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 -- 10.20.193.18:0/3812608940 shutdown_connections 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 --2- 10.20.193.18:0/3812608940 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1617d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 -- 10.20.193.18:0/3812608940 >> 10.20.193.18:0/3812608940 conn(0x7fc97c06d8c0 msgr2=0x7fc97c06dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 -- 10.20.193.18:0/3812608940 shutdown_connections 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.787+0000 7fc983f98640 1 -- 10.20.193.18:0/3812608940 wait complete. 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 10 monclient: monmap: 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7ff331267640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7ff331267640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7ff331267640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 librados: starting msgr at 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 librados: starting objecter 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 Processor -- start 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 -- start start 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 librados: setting wanted keys 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 1 librados: calling monclient init 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7fc983f98640 10 monclient: init 2026-02-24T18:35:33.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819339944 0 0) 0x7f7fd8159d80 con 0x7f7fd8153f90 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 10 monclient(hunting): my global_id is 4434 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.788+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd815a340 con 0x7f7fd8153f90 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-c con 0x7fc97c15a0f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a4d0 0x7fc97c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-a con 0x7fc97c15a4d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc97c160350 0x7fc97c15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): picked mon.noname-b con 0x7fc97c160350 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0d1e20 con 0x7fc97c15a4d0 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0c51a0 con 0x7fc97c160350 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0d2620 con 0x7fc97c15a0f0 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7fc983f98640 10 monclient(hunting): authenticate will time out at 1477.470081s 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7f0d91878640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 0x7f0d8c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.789+0000 7ffbd27fc640 1 --2- 10.20.193.18:0/3036568195 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd406f510 0x7ffbd415d810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098a0 tx=0x7ffbc802edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200eb933cd078791 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 10 monclient(hunting): handle_auth_done global_id 4437 payload 824 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 -- 10.20.193.18:0/2144418042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff32c15a310 msgr2=0x7ff32c1fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff32c15a310 0x7ff32c1fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 -- 10.20.193.18:0/2144418042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c06e0b0 msgr2=0x7ff32c1f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff32c06e0b0 0x7ff32c1f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.790+0000 7ff331267640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c1f8470 con 0x7ff32c1ee320 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 1 -- 10.20.193.18:0/1260801641 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e003c070 con 0x7f66ec1527f0 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: dump: 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.836 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 1 -- 10.20.193.18:0/1260801641 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e002fb40 con 0x7f66ec1527f0 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 1 -- 10.20.193.18:0/1260801641 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0037440 con 0x7f66ec1527f0 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 5 monclient: authenticate success, global_id 4423 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 4 set_mon_vals no callback set 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66e9ffb640 10 monclient: dump: 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 10 monclient: shutdown 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 -- 10.20.193.18:0/1260801641 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec1527f0 msgr2=0x7f66ec073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 --2- 10.20.193.18:0/1260801641 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec1527f0 0x7f66ec073530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06cdf0 tx=0x7f66e002eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 -- 10.20.193.18:0/1260801641 shutdown_connections 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 --2- 10.20.193.18:0/1260801641 >> v2:10.20.193.110:3300/0 conn(0x7f66ec15d730 0x7f66ec0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 --2- 10.20.193.18:0/1260801641 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec1527f0 0x7f66ec073530 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 -- 10.20.193.18:0/1260801641 >> 10.20.193.18:0/1260801641 conn(0x7f66ec0ce730 msgr2=0x7f66ec0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 -- 10.20.193.18:0/1260801641 shutdown_connections 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f66f16f5640 1 -- 10.20.193.18:0/1260801641 wait complete. 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 0x7f0d8c14acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41246/0 (socket says 10.20.193.18:41246) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 1 -- 10.20.193.18:0/1575185014 learned_addr learned my addr 10.20.193.18:0/1575185014 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 10 monclient(hunting): get_auth_request con 0x7f0d8c1527f0 auth_method 0 2026-02-24T18:35:33.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.791+0000 7f0d91878640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 10 monclient: monmap: 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f0d91878640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f0d91878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f0d91878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 librados: starting msgr at 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 librados: starting objecter 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 Processor -- start 2026-02-24T18:35:33.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7ff331267640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 0x7ff32c158df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b9a0 tx=0x7ff32800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7ff3217fa640 10 client.?.objecter ms_handle_connect 0x7ff32c1ee320 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7ff3217fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 -- start start 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 librados: setting wanted keys 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 1 librados: calling monclient init 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.792+0000 7f66f16f5640 10 monclient: init 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66ec1527f0 0x7f66ec0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-c con 0x7f66ec1527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f66ec15d2c0 0x7f66ec1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-a con 0x7f66ec15d2c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec15d730 0x7f66ec147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): picked mon.noname-b con 0x7f66ec15d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec074fa0 con 0x7f66ec15d2c0 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0747a0 con 0x7f66ec15d730 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec075120 con 0x7f66ec1527f0 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.793+0000 7f66f16f5640 10 monclient(hunting): authenticate will time out at 1477.474532s 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 -- 10.20.193.18:0/1575185014 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d8c15d2c0 msgr2=0x7f0d8c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d8c15d2c0 0x7f0d8c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 -- 10.20.193.18:0/1575185014 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d8c15d730 msgr2=0x7f0d8c157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d8c15d730 0x7f0d8c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.794+0000 7f0d91878640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0ca200 con 0x7f0d8c1527f0 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 1 -- 10.20.193.18:0/3668237198 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7802fc70 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: dump: 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.840 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 1 -- 10.20.193.18:0/3668237198 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7802fe10 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 1 -- 10.20.193.18:0/3668237198 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78037610 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8129f640 10 monclient: dump: 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 5 monclient: authenticate success, global_id 4421 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 4 set_mon_vals no callback set 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 10 monclient: shutdown 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 -- 10.20.193.18:0/3668237198 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c0d1ea0 msgr2=0x7f0b7c0c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3668237198 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0ce0a0 tx=0x7f0b7802f1c0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 -- 10.20.193.18:0/3668237198 shutdown_connections 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3668237198 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c0c9b00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 1 -- 10.20.193.18:0/2208805029 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec003c070 con 0x7f3ed01527f0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: dump: 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 1 -- 10.20.193.18:0/2208805029 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec002fb60 con 0x7f3ed01527f0 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ed72f1640 5 monclient: authenticate success, global_id 4427 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ed72f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ed72f1640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 1 -- 10.20.193.18:0/2208805029 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0037440 con 0x7f3ed01527f0 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ed72f1640 4 set_mon_vals no callback set 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f3ebffff640 10 monclient: dump: 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3668237198 >> v2:10.20.193.32:3300/0 conn(0x7f0b7c0ceca0 0x7f0b7c0d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f5b0d4cb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b081f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3668237198 >> v2:10.20.193.18:3300/0 conn(0x7f0b7c15f650 0x7f0b7c15fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 -- 10.20.193.18:0/3668237198 >> 10.20.193.18:0/3668237198 conn(0x7f0b7c072fc0 msgr2=0x7f0b7c0733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 -- 10.20.193.18:0/3668237198 shutdown_connections 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.796+0000 7f0b8452c640 1 -- 10.20.193.18:0/3668237198 wait complete. 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 10 monclient: shutdown 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 -- 10.20.193.18:0/2208805029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 msgr2=0x7f3ed0073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 --2- 10.20.193.18:0/2208805029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 0x7f3ed0073530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009970 tx=0x7f3ec002eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478004480 con 0x7fa4841527f0 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: dump: 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa478010070 con 0x7fa4841527f0 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 5 monclient: authenticate success, global_id 4424 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 monclient: _renew_subs 2026-02-24T18:35:33.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48415dd90 con 0x7fa4841527f0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 client.4424.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 monclient: _renew_subs 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48415e350 con 0x7fa4841527f0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 1 librados: init done 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 -- 10.20.193.18:0/2208805029 shutdown_connections 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 --2- 10.20.193.18:0/2208805029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 0x7f3ed0073530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 -- 10.20.193.18:0/2208805029 >> 10.20.193.18:0/2208805029 conn(0x7f3ed00ce730 msgr2=0x7f3ed00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478002920 con 0x7fa4841527f0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 -- 10.20.193.18:0/2208805029 shutdown_connections 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4817fa640 4 set_mon_vals no callback set 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa45bfff640 10 monclient: dump: 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 1 -- 10.20.193.18:0/2208805029 wait complete. 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b081f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44104/0 (socket says 10.20.193.18:44104) 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/161218493 learned_addr learned my addr 10.20.193.18:0/161218493 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 10 monclient(hunting): get_auth_request con 0x7f5b0815fad0 auth_method 0 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f5b0d4cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 10 monclient: monmap: 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 librados: starting msgr at 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 client.4424.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 15 client.4424.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 20 client.4424.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 client.4424.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa48415db40 con 0x7fa4841527f0 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 librados: starting objecter 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0d91878640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 0x7f0d8c14acc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b7b0 tx=0x7f0d7c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0d7a7fc640 10 client.?.objecter ms_handle_connect 0x7f0d8c1527f0 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0d7a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 Processor -- start 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 -- start start 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 librados: setting wanted keys 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 1 librados: calling monclient init 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f0b8452c640 10 monclient: init 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.797+0000 7f3ed72f1640 10 monclient: monmap: 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 librados: starting msgr at 2026-02-24T18:35:33.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 librados: starting objecter 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b7c0ceca0 0x7f0b7c1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-a con 0x7f0b7c0ceca0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 Processor -- start 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-b con 0x7f0b7c0d1ea0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c15f650 0x7f0b7c1e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): picked mon.noname-c con 0x7f0b7c15f650 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cf0f0 con 0x7f0b7c0ceca0 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0c95e0 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1e7390 con 0x7f0b7c15f650 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f0b8452c640 10 monclient(hunting): authenticate will time out at 1477.479286s 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 -- start start 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 librados: setting wanted keys 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 librados: calling monclient init 2026-02-24T18:35:33.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient: init 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 0x7fa3bc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fc981d0d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 0x7fa3bc0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44108/0 (socket says 10.20.193.18:44108) 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 1 -- 10.20.193.18:0/1779742788 learned_addr learned my addr 10.20.193.18:0/1779742788 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 10 monclient(hunting): get_auth_request con 0x7fa3bc1527f0 auth_method 0 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7fa3c1928640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 0x7f3ed01f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-c con 0x7f3ed01527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 0x7f3ed014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-b con 0x7f3ed015d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3ed015d730 0x7f3ed00c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): picked mon.noname-a con 0x7f3ed015d730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0074fa0 con 0x7f3ed015d730 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00747a0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0075120 con 0x7f3ed01527f0 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.798+0000 7f3ed72f1640 10 monclient(hunting): authenticate will time out at 1477.479693s 2026-02-24T18:35:33.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7fd118a60640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 0x7fd11414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 10 monclient(hunting): handle_auth_done global_id 4430 payload 824 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b081527f0 msgr2=0x7f5b08149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5b081527f0 0x7f5b08149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/161218493 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b08073530 msgr2=0x7f5b081f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5b08073530 0x7f5b081f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.799+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080c7780 con 0x7f5b0815fad0 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44130/0 (socket says 10.20.193.18:44130) 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 1 -- 10.20.193.18:0/2674794266 learned_addr learned my addr 10.20.193.18:0/2674794266 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): get_auth_request con 0x7fc97c15a0f0 auth_method 0 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7faa18024060 con 0x7faa2406ff20 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fa3c1928640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fa3c1928640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fa3c1928640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7faa037fe640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa9f0078e50 0x7fa9f007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7fc981d0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7faa22575640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa9f0078e50 0x7fa9f007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.800+0000 7faa22575640 10 monclient: get_auth_request con 0x7fa9f0078e50 auth_method 0 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 0x7fd11414a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41294/0 (socket says 10.20.193.18:41294) 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 1 -- 10.20.193.18:0/3318530097 learned_addr learned my addr 10.20.193.18:0/3318530097 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): get_auth_request con 0x7fd11415fa30 auth_method 0 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7fd118a60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7faa037fe640 10 client.4414.objecter ms_handle_connect 0x7fa9f0078e50 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.801+0000 7faa22575640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa9f0078e50 0x7fa9f007b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa0c039000 tx=0x7faa0c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 10 monclient(hunting): handle_auth_done global_id 4433 payload 824 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 -- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa3bc15d730 msgr2=0x7fa3bc1f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa3bc15d730 0x7fa3bc1f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 -- 10.20.193.18:0/1779742788 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa3bc15d2c0 msgr2=0x7fa3bc1f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa3bc15d2c0 0x7fa3bc1f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7fa3c1928640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc1f7d20 con 0x7fa3bc1527f0 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7faa1805de20 con 0x7faa2406ff20 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 10 client.4414.objecter ms_dispatch2 0x7faa24000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 3 client.4414.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 3 client.4414.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 10 client.4414.objecter checking command 1 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fa9f0081390 0x7fa9f00837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter _get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 15 client.4414.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter _get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 15 client.4414.objecter _session_command_op_assign 0 1 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 4 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 10 client.4414.objecter _send_command 1 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa18016ec0 con 0x7fa9f0081390 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.802+0000 7faa037fe640 20 client.4414.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa1805e1e0 con 0x7faa2406ff20 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7faa037fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7faa23577640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fa9f0081390 0x7fa9f00837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7faa23577640 10 monclient: get_auth_request con 0x7fa9f0081390 auth_method 0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 1 -- 10.20.193.18:0/1722328463 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c003c070 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: dump: 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 1 -- 10.20.193.18:0/1722328463 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c002faf0 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 1 -- 10.20.193.18:0/1722328463 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0037440 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d099e640 10 monclient: dump: 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d3c2b640 5 monclient: authenticate success, global_id 4417 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d3c2b640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.803+0000 7fa0d3c2b640 4 set_mon_vals no callback set 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 10 monclient: shutdown 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1722328463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 msgr2=0x7fa0cc15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1722328463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 0x7fa0cc15f3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc08f510 tx=0x7fa0c002ede0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 10 monclient(hunting): handle_auth_done global_id 4436 payload 824 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 -- 10.20.193.18:0/2674794266 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc97c160350 msgr2=0x7fc97c15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc97c160350 0x7fc97c15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 -- 10.20.193.18:0/2674794266 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a4d0 msgr2=0x7fc97c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc97c15a4d0 0x7fc97c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fc981d0d640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c1fc6c0 con 0x7fc97c15a0f0 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1722328463 shutdown_connections 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1722328463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 0x7fa0cc15f3c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1722328463 >> v2:10.20.193.18:3300/0 conn(0x7fa0cc1527f0 0x7fa0cc073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1722328463 >> 10.20.193.18:0/1722328463 conn(0x7fa0cc0ce730 msgr2=0x7fa0cc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1722328463 shutdown_connections 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.804+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1722328463 wait complete. 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 10 monclient(hunting): handle_auth_done global_id 4446 payload 824 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 -- 10.20.193.18:0/3318530097 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1140d1ea0 msgr2=0x7fd1140ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1140d1ea0 0x7fd1140ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 -- 10.20.193.18:0/3318530097 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd11415f650 msgr2=0x7fd1141f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd11415f650 0x7fd1141f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fd118a60640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd114157040 con 0x7fd11415fa30 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 10 monclient: monmap: 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7f5b0d4cb640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b081f4d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b04039040 tx=0x7f5b04004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7f5afd7fa640 10 client.?.objecter ms_handle_connect 0x7f5b0815fad0 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7f5afd7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 librados: starting msgr at 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 librados: starting objecter 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 Processor -- start 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7faa037fe640 10 client.4414.objecter ms_handle_connect 0x7fa9f0081390 2026-02-24T18:35:33.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7faa23577640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fa9f0081390 0x7fa9f00837d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 -- start start 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 librados: setting wanted keys 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 1 librados: calling monclient init 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.805+0000 7fa0d3c2b640 10 monclient: init 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 0x7fa0cc0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-c con 0x7fa0cc1527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 0x7fa0cc148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-b con 0x7fa0cc15d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0cc15d730 0x7fa0cc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): picked mon.noname-a con 0x7fa0cc15d730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc074fa0 con 0x7fa0cc15d730 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc0747a0 con 0x7fa0cc15d2c0 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc075120 con 0x7fa0cc1527f0 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa0d3c2b640 10 monclient(hunting): authenticate will time out at 1477.487206s 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa3c1928640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 0x7fa3bc0c7db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b780 tx=0x7fa3ac00bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa3aa7fc640 10 client.?.objecter ms_handle_connect 0x7fa3bc1527f0 2026-02-24T18:35:33.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fa3aa7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fd118a60640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 0x7fd11414a280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd10802f6f0 tx=0x7fd108004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fd110ff9640 10 client.?.objecter ms_handle_connect 0x7fd11415fa30 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.806+0000 7fd110ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7fc981d0d640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1fc180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0099d0 tx=0x7fc96c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7fc96a7fc640 10 client.?.objecter ms_handle_connect 0x7fc97c15a0f0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7fc96a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 10 monclient(hunting): handle_auth_done global_id 4432 payload 274 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 1 -- 10.20.193.18:0/1630791889 >> v1:10.20.193.110:6789/0 conn(0x7f462c15f900 legacy=0x7f462c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 1 -- 10.20.193.18:0/1630791889 >> v1:10.20.193.18:6789/0 conn(0x7f462c1527f0 legacy=0x7f462c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f462affd640 1 -- 10.20.193.18:0/1630791889 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c15d9d0 con 0x7f462c073530 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32800c790 con 0x7ff32c1ee320 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: dump: 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.856 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff328010070 con 0x7ff32c1ee320 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff3217fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7ff322ffd640 4 set_mon_vals no callback set 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.807+0000 7f66ea7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f66ec15d2c0 0x7f66ec1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32800cb90 con 0x7ff32c1ee320 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 5 monclient: authenticate success, global_id 4437 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 monclient: _renew_subs 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c1f8750 con 0x7ff32c1ee320 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff3217fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff3217fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff3217fa640 10 monclient: dump: 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.857 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 client.4437.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 monclient: _renew_subs 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c1eb8f0 con 0x7ff32c1ee320 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 1 librados: init done 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.808+0000 7ff332cf1640 10 client.4437.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7ff332cf1640 15 client.4437.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7ff332cf1640 20 client.4437.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7ff332cf1640 10 client.4437.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7ff332cf1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff32c06e4b0 con 0x7ff32c1ee320 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.809+0000 7f0b81aa0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53126/0 (socket says 10.20.193.18:53126) 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 1 -- 10.20.193.18:0/3994694550 learned_addr learned my addr 10.20.193.18:0/3994694550 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 10 monclient(hunting): get_auth_request con 0x7f0b7c0d1ea0 auth_method 0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f0b81aa0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 1 -- 10.20.193.18:0/2004343576 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc003c070 con 0x7fecd0155eb0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: dump: 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 1 -- 10.20.193.18:0/2004343576 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc002fb60 con 0x7fecd0155eb0 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 1 -- 10.20.193.18:0/2004343576 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0037440 con 0x7fecd0155eb0 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecd79f7640 5 monclient: authenticate success, global_id 4420 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecd79f7640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecd79f7640 4 set_mon_vals no callback set 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7fecbffff640 10 monclient: dump: 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 1 -- 10.20.193.18:0/3036568195 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc803c070 con 0x7ffbd406f510 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: dump: 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 1 -- 10.20.193.18:0/3036568195 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc802fae0 con 0x7ffbd406f510 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 1 -- 10.20.193.18:0/3036568195 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8037440 con 0x7ffbd406f510 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd9334640 5 monclient: authenticate success, global_id 4426 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd9334640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd9334640 4 set_mon_vals no callback set 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7ffbd1ffb640 10 monclient: dump: 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f66ec15d2c0 0x7f66ec1f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41302/0 (socket says 10.20.193.18:41302) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 1 -- 10.20.193.18:0/3469857683 learned_addr learned my addr 10.20.193.18:0/3469857683 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 10 monclient(hunting): get_auth_request con 0x7f66ec15d2c0 auth_method 0 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.810+0000 7f66ea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 10 monclient: shutdown 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 -- 10.20.193.18:0/2004343576 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 msgr2=0x7fecd015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 --2- 10.20.193.18:0/2004343576 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd015d1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009970 tx=0x7fecc002eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 10 monclient: shutdown 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 -- 10.20.193.18:0/3036568195 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd406f510 msgr2=0x7ffbd415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 --2- 10.20.193.18:0/3036568195 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd406f510 0x7ffbd415d810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098a0 tx=0x7ffbc802edd0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 -- 10.20.193.18:0/3036568195 shutdown_connections 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 --2- 10.20.193.18:0/3036568195 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd406f510 0x7ffbd415d810 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 -- 10.20.193.18:0/3036568195 >> 10.20.193.18:0/3036568195 conn(0x7ffbd40cc190 msgr2=0x7ffbd40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 -- 10.20.193.18:0/3036568195 shutdown_connections 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 -- 10.20.193.18:0/3036568195 wait complete. 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 -- 10.20.193.18:0/2004343576 shutdown_connections 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 --2- 10.20.193.18:0/2004343576 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd015d1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 -- 10.20.193.18:0/2004343576 >> 10.20.193.18:0/2004343576 conn(0x7fecd00cf580 msgr2=0x7fecd00cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f0b81aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f0b81aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f0b81aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 -- 10.20.193.18:0/2004343576 shutdown_connections 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7fecd79f7640 1 -- 10.20.193.18:0/2004343576 wait complete. 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 10 monclient: monmap: 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.861 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754670167 0 0) 0x7f7fd815a060 con 0x7f7fd80cc700 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 10 monclient(hunting): my global_id is 4429 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd8159d80 con 0x7f7fd80cc700 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 librados: starting msgr at 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.811+0000 7ffbd9334640 1 librados: starting objecter 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 Processor -- start 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 -- start start 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 librados: setting wanted keys 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 librados: calling monclient init 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient: init 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 10 monclient: monmap: 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.862 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 librados: starting msgr at 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 librados: starting objecter 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 Processor -- start 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7f0b822a1640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b7c0ceca0 0x7f0b7c1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbd406f510 0x7ffbd41ea380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-c con 0x7ffbd406f510 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd40cb0a0 0x7ffbd40ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-b con 0x7ffbd40cb0a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 0x7ffbd41f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): picked mon.noname-a con 0x7ffbd40cccb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40cbb90 con 0x7ffbd40cccb0 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40707f0 con 0x7ffbd40cb0a0 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40cbd10 con 0x7ffbd406f510 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7ffbd9334640 10 monclient(hunting): authenticate will time out at 1477.493588s 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 -- start start 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 librados: setting wanted keys 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 1 librados: calling monclient init 2026-02-24T18:35:33.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7fecd79f7640 10 monclient: init 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7f0b822a1640 10 monclient(hunting): get_auth_request con 0x7f0b7c0ceca0 auth_method 0 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7f0b822a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7f0b822a1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.812+0000 7f0b822a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fecd0070210 0x7fecd01ea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-a con 0x7fecd0070210 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd00c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-b con 0x7fecd0155eb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fecd015c430 0x7fecd00c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): picked mon.noname-c con 0x7fecd015c430 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fecd015d6e0 con 0x7fecd0070210 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fecd006fd80 con 0x7fecd0155eb0 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0156290 con 0x7fecd015c430 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7fecd79f7640 10 monclient(hunting): authenticate will time out at 1477.494114s 2026-02-24T18:35:33.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b822a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b822a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b822a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 10 monclient(hunting): handle_auth_done global_id 4435 payload 824 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 -- 10.20.193.18:0/3994694550 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c15f650 msgr2=0x7f0b7c1e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b7c15f650 0x7f0b7c1e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 -- 10.20.193.18:0/3994694550 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b7c0ceca0 msgr2=0x7f0b7c1f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b7c0ceca0 0x7f0b7c1f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.813+0000 7f0b81aa0640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c1f3d80 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.814+0000 7f66ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.814+0000 7f66ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.814+0000 7f66ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.814+0000 7f3ed5867640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3ed015d730 0x7f3ed00c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.815+0000 7f0b81aa0640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c143940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b500 tx=0x7f0b7000b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.815+0000 7f0b62ffd640 10 client.?.objecter ms_handle_connect 0x7f0b7c0d1ea0 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.815+0000 7f0b62ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eb7fe640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec15d730 0x7f66ec147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638380911 0 0) 0x7f7fc4003620 con 0x7f7fd8153bb0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 >> v1:10.20.193.32:6789/0 conn(0x7f7fd80cc700 legacy=0x7f7fd80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 >> v1:10.20.193.18:6789/0 conn(0x7f7fd8153f90 legacy=0x7f7fd8074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd815a620 con 0x7f7fd8153bb0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f7fc8002f00 con 0x7f7fd8153bb0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: dump: 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 >> v1:10.20.193.110:6789/0 conn(0x7f7fd8153bb0 legacy=0x7f7fd80cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4003fb0 0x7f7fc4004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): picked mon.c con 0x7f7fc4003fb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fc40044c0 0x7f7fc4004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): picked mon.a con 0x7f7fc40044c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 0x7f7fc400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): picked mon.b con 0x7f7fc4009ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc400c490 con 0x7f7fc40044c0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc400c610 con 0x7f7fc4009ab0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc400c810 con 0x7f7fc4003fb0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f7fcf7fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eb7fe640 10 monclient(hunting): get_auth_request con 0x7f66ec15d730 auth_method 0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f3ed4865640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 0x7f3ed014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c046020 con 0x7fc97c15a0f0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: dump: 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.867 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96c042700 con 0x7fc97c15a0f0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c03c040 con 0x7fc97c15a0f0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96a7fc640 10 monclient: dump: 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc96bfff640 4 set_mon_vals no callback set 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 5 monclient: authenticate success, global_id 4436 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 10 monclient: _renew_subs 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1f87d0 con 0x7fc97c15a0f0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 10 client.4436.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 10 monclient: _renew_subs 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c1f8d90 con 0x7fc97c15a0f0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7fc983f98640 1 librados: init done 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.816+0000 7f66eaffd640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66ec1527f0 0x7f66ec0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): get_auth_request con 0x7f66ec1527f0 auth_method 0 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 10 client.4436.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 15 client.4436.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 20 client.4436.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 10 client.4436.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc97c0c4fc0 con 0x7fc97c15a0f0 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.817+0000 7f66eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 0x7f3ed014c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53128/0 (socket says 10.20.193.18:53128) 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 1 -- 10.20.193.18:0/3335183108 learned_addr learned my addr 10.20.193.18:0/3335183108 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): get_auth_request con 0x7f3ed015d2c0 auth_method 0 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed5867640 10 monclient(hunting): get_auth_request con 0x7f3ed015d730 auth_method 0 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed5867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed5867640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed5867640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70004480 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: dump: 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70014070 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70002920 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 5 monclient: authenticate success, global_id 4435 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 10 monclient: _renew_subs 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b80a9e640 4 set_mon_vals no callback set 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c1f4050 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b62ffd640 10 monclient: dump: 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 10 client.4435.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 10 monclient: _renew_subs 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1f4610 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f0b8452c640 1 librados: init done 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.818+0000 7f3ed4865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 10 client.4435.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 15 client.4435.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 20 client.4435.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 10 client.4435.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b7c144ca0 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 10 monclient(hunting): handle_auth_done global_id 4439 payload 824 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 -- 10.20.193.18:0/3469857683 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec15d730 msgr2=0x7f66ec147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66ec15d730 0x7f66ec147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 -- 10.20.193.18:0/3469857683 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f66ec15d2c0 msgr2=0x7f66ec1f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f66ec15d2c0 0x7f66ec1f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.819+0000 7f66eaffd640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0c8b90 con 0x7f66ec1527f0 2026-02-24T18:35:33.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7ffbd37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 0x7ffbd41f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04046020 con 0x7f5b0815fad0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 10 monclient: dump: 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac004480 con 0x7fa3bc1527f0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7fa3aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7fa3aa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7fa3aa7fc640 10 monclient: dump: 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.821+0000 7f5afd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b04037de0 con 0x7f5b0815fad0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 5 monclient: authenticate success, global_id 4430 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f66eaffd640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66ec1527f0 0x7f66ec0ca1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 monclient: _renew_subs 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f66cb7fe640 10 client.?.objecter ms_handle_connect 0x7f66ec1527f0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f66cb7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ac010070 con 0x7fa3bc1527f0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3abfff640 4 set_mon_vals no callback set 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac002920 con 0x7fa3bc1527f0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b080c7a60 con 0x7f5b0815fad0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3aa7fc640 10 monclient: dump: 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.873 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 client.4430.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04041690 con 0x7f5b0815fad0 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afd7fa640 10 monclient: dump: 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5afeffd640 4 set_mon_vals no callback set 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 monclient: _renew_subs 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b080c8020 con 0x7f5b0815fad0 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 1 librados: init done 2026-02-24T18:35:33.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 0x7fa0cc0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 5 monclient: authenticate success, global_id 4433 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 10 monclient: _renew_subs 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc1f45a0 con 0x7fa3bc1527f0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 10 client.4433.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 10 monclient: _renew_subs 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc0c72d0 con 0x7fa3bc1527f0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 10 monclient(hunting): handle_auth_done global_id 4441 payload 824 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 -- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 msgr2=0x7f3ed01f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3ed01527f0 0x7f3ed01f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 -- 10.20.193.18:0/3335183108 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3ed015d730 msgr2=0x7f3ed00c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3ed015d730 0x7f3ed00c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f3ed4865640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed01f55c0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa3c3bb3640 1 librados: init done 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 client.4430.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 15 client.4430.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 20 client.4430.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 client.4430.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b080c7530 con 0x7f5b0815fad0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 0x7fa0cc0c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44166/0 (socket says 10.20.193.18:44166) 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 1 -- 10.20.193.18:0/1784390635 learned_addr learned my addr 10.20.193.18:0/1784390635 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 10 monclient(hunting): get_auth_request con 0x7fa0cc1527f0 auth_method 0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.822+0000 7fa0d19a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 10 client.4433.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 15 client.4433.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 20 client.4433.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 10 client.4433.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3bc0c86e0 con 0x7fa3bc1527f0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 0x7ffbd41f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41340/0 (socket says 10.20.193.18:41340) 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 1 -- 10.20.193.18:0/2278326037 learned_addr learned my addr 10.20.193.18:0/2278326037 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 10 monclient(hunting): get_auth_request con 0x7ffbd40cccb0 auth_method 0 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7ffbd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa0d19a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.823+0000 7fa0d19a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.824+0000 7fa0d19a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.824+0000 7ffbd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.824+0000 7ffbd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.824+0000 7ffbd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 -- 10.20.193.18:0/2278326037 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbd406f510 msgr2=0x7ffbd41ea380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbd406f510 0x7ffbd41ea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 -- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd40cb0a0 msgr2=0x7ffbd40ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbd40cb0a0 0x7ffbd40ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7ffbd37fe640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41ea8c0 con 0x7ffbd40cccb0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== osd.0 v2:10.20.193.18:6817/1076077393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7faa18016ec0 con 0x7fa9f0081390 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7faa037fe640 10 client.4414.objecter ms_dispatch2 0x7faa24000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7faa037fe640 10 client.4414.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7faa037fe640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 4 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.827+0000 7faa037fe640 15 client.4414.objecter _session_command_op_remove 0 1 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004480 con 0x7f0d8c1527f0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: dump: 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.877 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c010070 con 0x7f0d8c1527f0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c002920 con 0x7f0d8c1527f0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 5 monclient: authenticate success, global_id 4440 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 10 monclient: _renew_subs 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7bfff640 4 set_mon_vals no callback set 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c0ca4e0 con 0x7f0d8c1527f0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 10 client.4440.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 10 monclient: _renew_subs 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c156a40 con 0x7f0d8c1527f0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d93b03640 1 librados: init done 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f0d7a7fc640 10 monclient: dump: 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7fa0d21a1640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0cc15d730 0x7fa0cc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f3ed4865640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 0x7f3ed014c240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b5a0 tx=0x7f3ec400ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f3ebdffb640 10 client.?.objecter ms_handle_connect 0x7f3ed015d2c0 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.828+0000 7f3ebdffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 10 client.4440.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 15 client.4440.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 20 client.4440.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 10 client.4440.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d8c077010 con 0x7f0d8c1527f0 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108046020 con 0x7fd11415fa30 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: dump: 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd108041450 con 0x7fd11415fa30 2026-02-24T18:35:33.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 5 monclient: authenticate success, global_id 4446 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 10 monclient: _renew_subs 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd114157320 con 0x7fd11415fa30 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108041840 con 0x7fd11415fa30 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd1127fc640 4 set_mon_vals no callback set 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd110ff9640 10 monclient: dump: 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 10 client.4446.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 10 monclient: _renew_subs 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141578e0 con 0x7fd11415fa30 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.829+0000 7fd11a4ea640 1 librados: init done 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 10 client.4446.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 15 client.4446.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 20 client.4446.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 10 client.4446.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd114160530 con 0x7fd11415fa30 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7ffbd37fe640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 0x7ffbd41f43f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c490 tx=0x7ffbbc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7ffbb37fe640 10 client.?.objecter ms_handle_connect 0x7ffbd40cccb0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.830+0000 7ffbb37fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): get_auth_request con 0x7fa0cc15d730 auth_method 0 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa47801d030 con 0x7fa4841527f0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d21a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fecd576c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fecd0070210 0x7fecd01ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa45bfff640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa454078e50 0x7fa45407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa4827fc640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa454078e50 0x7fa45407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa4827fc640 10 monclient: get_auth_request con 0x7fa454078e50 auth_method 0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 10 monclient(hunting): handle_auth_done global_id 4442 payload 824 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 -- 10.20.193.18:0/1784390635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 msgr2=0x7fa0cc148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0cc15d2c0 0x7fa0cc148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 -- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0cc15d730 msgr2=0x7fa0cc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0cc15d730 0x7fa0cc15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.831+0000 7fa0d19a0640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc0c6de0 con 0x7fa0cc1527f0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.832+0000 7fa4827fc640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa454078e50 0x7fa45407b310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa48406cdf0 tx=0x7fa46c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.832+0000 7fa45bfff640 10 client.4424.objecter ms_handle_connect 0x7fa454078e50 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 10 client.4414.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 15 client.4414.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 20 client.4414.objecter _get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 20 client.4414.objecter _get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 15 client.4414.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 20 client.4414.objecter get_session s=0x7fa9f0080cc0 osd=0 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 15 client.4414.objecter _session_command_op_assign 0 2 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 4 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 10 client.4414.objecter _send_command 2 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.833+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 --> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa2414f1f0 con 0x7fa9f0081390 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.836+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0b70019460 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.836+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa47801de30 con 0x7fa4841527f0 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.836+0000 7fa45bfff640 10 client.4424.objecter ms_dispatch2 0x7fa484000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.836+0000 7fa45bfff640 3 client.4424.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.836+0000 7fa45bfff640 3 client.4424.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 10 client.4424.objecter checking command 1 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7fa454081390 0x7fa4540837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter _get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 15 client.4424.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter _get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 15 client.4424.objecter _session_command_op_assign 6 1 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 4 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 10 client.4424.objecter _send_command 1 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa478021ec0 con 0x7fa454081390 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 20 client.4424.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7f0b62ffd640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b4c078e50 0x7f0b4c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa47805d7b0 con 0x7fa4841527f0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa45bfff640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa037fe640 1 -- 10.20.193.18:0/2857618340 <== osd.0 v2:10.20.193.18:6817/1076077393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa2414f1f0 con 0x7fa9f0081390 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa037fe640 10 client.4414.objecter ms_dispatch2 0x7faa24000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa037fe640 10 client.4414.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa037fe640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 4 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa037fe640 15 client.4414.objecter _session_command_op_remove 0 2 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7f0b822a1640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b4c078e50 0x7f0b4c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7f0b822a1640 10 monclient: get_auth_request con 0x7f0b4c078e50 auth_method 0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa4837fe640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7fa454081390 0x7fa4540837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 10 librados: watch_flush enter 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 10 librados: watch_flush exit 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 10 client.4414.objecter close_session for osd.0 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fa9f0081390 msgr2=0x7fa9f00837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fa9f0081390 0x7fa9f00837d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 20 client.4414.objecter put_session s=0x7fa9f0080cc0 osd=0 1 2026-02-24T18:35:33.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 20 client.4414.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 10 client.4414.objecter successfully canceled tick 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa9f0078e50 msgr2=0x7fa9f007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa9f0078e50 0x7fa9f007b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa0c039000 tx=0x7faa0c031000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 10 monclient: shutdown 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 msgr2=0x7faa241f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 --2- 10.20.193.18:0/2857618340 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa2406ff20 0x7faa241f92c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa1800ebe0 tx=0x7faa1800c520 comp rx=0 tx=0).stop 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa23577640 1 -- 10.20.193.18:0/2857618340 reap_dead start 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7fa4837fe640 10 monclient: get_auth_request con 0x7fa454081390 auth_method 0 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 shutdown_connections 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 >> 10.20.193.18:0/2857618340 conn(0x7faa240cd780 msgr2=0x7faa240cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 shutdown_connections 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 -- 10.20.193.18:0/2857618340 wait complete. 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.837+0000 7faa29196640 1 librados: shutdown 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.838+0000 7f0b822a1640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b4c078e50 0x7f0b4c07b310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006fd0 tx=0x7f0b6c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.838+0000 7f0b62ffd640 10 client.4435.objecter ms_handle_connect 0x7f0b4c078e50 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.838+0000 7f7fdcb58640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4003fb0 0x7f7fc4004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.838+0000 7fecd5f6d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fecd015c430 0x7fecd00c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0b7005dc00 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 10 client.4435.objecter ms_dispatch2 0x7f0b7c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 3 client.4435.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 3 client.4435.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 10 client.4435.objecter checking command 1 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f0b4c0813f0 0x7f0b4c083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter _get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 15 client.4435.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter _get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 15 client.4435.objecter _session_command_op_assign 15 1 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 4 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 10 client.4435.objecter _send_command 1 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b4c083f00 con 0x7f0b4c0813f0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.839+0000 7f0b62ffd640 20 client.4435.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b7005dfc0 con 0x7f0b7c0d1ea0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7f0b62ffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7f0b82aa2640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f0b4c0813f0 0x7f0b4c083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7fa45bfff640 10 client.4424.objecter ms_handle_connect 0x7fa454081390 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7fa4837fe640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7fa454081390 0x7fa4540837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.840+0000 7f0b82aa2640 10 monclient: get_auth_request con 0x7f0b4c0813f0 auth_method 0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.842+0000 7f0b82aa2640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f0b4c0813f0 0x7f0b4c083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.842+0000 7f0b62ffd640 10 client.4435.objecter ms_handle_connect 0x7f0b4c0813f0 2026-02-24T18:35:33.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.844+0000 7f462affd640 1 --2- 10.20.193.18:0/1630791889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c073530 0x7f462c15f3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f461c0098a0 tx=0x7f461c02ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf8bc8c68653524 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4012070 con 0x7f3ed015d2c0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: dump: 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec40044a0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40029c0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebdffb640 10 monclient: dump: 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ebf7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 5 monclient: authenticate success, global_id 4441 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 10 monclient: _renew_subs 2026-02-24T18:35:33.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed01f7b80 con 0x7f3ed015d2c0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 10 client.4441.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 10 monclient: _renew_subs 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01f3e00 con 0x7f3ed015d2c0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.845+0000 7f3ed72f1640 1 librados: init done 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f7fdcb58640 10 monclient(hunting): get_auth_request con 0x7f7fc4003fb0 auth_method 0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f7fdcb58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f7fdcb58640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f7fdcb58640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 10 client.4441.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 15 client.4441.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 20 client.4441.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 10 client.4441.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.846+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ed00d29e0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fecd015c430 0x7fecd00c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44180/0 (socket says 10.20.193.18:44180) 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 1 -- 10.20.193.18:0/2719163345 learned_addr learned my addr 10.20.193.18:0/2719163345 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 10 monclient(hunting): get_auth_request con 0x7fecd015c430 auth_method 0 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.847+0000 7fecd5f6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:33.888 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7fecd4f6b640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd00c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7f7fdcb58640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7f7fdcb58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7f7fdcb58640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7fecd4f6b640 10 monclient(hunting): get_auth_request con 0x7fecd0155eb0 auth_method 0 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7fecd4f6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7fecd4f6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.848+0000 7fecd4f6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.849+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0d7c01d030 con 0x7f0d8c1527f0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.849+0000 7fecd576c640 10 monclient(hunting): get_auth_request con 0x7fecd0070210 auth_method 0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.849+0000 7fecd576c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.849+0000 7fecd576c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.849+0000 7fecd576c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f0d7a7fc640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d5c078e80 0x7f0d5c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f0d91077640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d5c078e80 0x7f0d5c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f0d91077640 10 monclient: get_auth_request con 0x7f0d5c078e80 auth_method 0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7fecd4f6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7fecd4f6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7fecd4f6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f0d7a7fc640 10 client.4440.objecter ms_handle_connect 0x7f0d5c078e80 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f0d91077640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d5c078e80 0x7f0d5c07b340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006fd0 tx=0x7f0d80006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.850+0000 7f7fdd359640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 0x7f7fc400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 10 monclient(hunting): handle_auth_done global_id 4444 payload 824 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 -- 10.20.193.18:0/2719163345 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fecd015c430 msgr2=0x7fecd00c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fecd015c430 0x7fecd00c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 -- 10.20.193.18:0/2719163345 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fecd0070210 msgr2=0x7fecd01ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fecd0070210 0x7fecd01ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7fecd4f6b640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00c6c40 con 0x7fecd0155eb0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7f7fdd359640 10 monclient(hunting): get_auth_request con 0x7f7fc4009ab0 auth_method 0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7f7fdd359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7f7fdd359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.852+0000 7f7fdd359640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.853+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.853+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.853+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 10 monclient(hunting): handle_auth_done global_id 4418 payload 293 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 -- 10.20.193.18:0/2835151145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4003fb0 msgr2=0x7f7fc4004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4003fb0 0x7f7fc4004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 -- 10.20.193.18:0/2835151145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fc40044c0 msgr2=0x7f7fc4004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fc40044c0 0x7f7fc4004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 monclient: found mon.b 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 -- 10.20.193.18:0/2835151145 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4003620 con 0x7f7fc4009ab0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdede3640 5 monclient: authenticate success, global_id 4418 2026-02-24T18:35:33.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdede3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7fecd4f6b640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd00c7fc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecc400b470 tx=0x7fecc400b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7fecbdffb640 10 client.?.objecter ms_handle_connect 0x7fecd0155eb0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7fecbdffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0d7c05d550 con 0x7f0d8c1527f0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 10 client.4440.objecter ms_dispatch2 0x7f0d8c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 3 client.4440.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 3 client.4440.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f7fdd359640 1 -- 10.20.193.18:0/2835151145 reap_dead start 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 10 client.4440.objecter checking command 1 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f0d5c0813c0 0x7f0d5c083800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter _get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 15 client.4440.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter _get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 15 client.4440.objecter _session_command_op_assign 12 1 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 4 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 10 client.4440.objecter _send_command 1 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d7c021ec0 con 0x7f0d5c0813c0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 20 client.4440.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d7c05d910 con 0x7f0d8c1527f0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.854+0000 7f0d7a7fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.855+0000 7f0d92079640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f0d5c0813c0 0x7f0d5c083800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.855+0000 7f0d92079640 10 monclient: get_auth_request con 0x7f0d5c0813c0 auth_method 0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.855+0000 7f7fdd359640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 0x7f7fc400bea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400c320 tx=0x7f7fd402f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06b7fb76171483b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.856+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa3ac01d030 con 0x7fa3bc1527f0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.856+0000 7fa3aa7fc640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa38c078e50 0x7fa38c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.856+0000 7fa3c1127640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa38c078e50 0x7fa38c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.856+0000 7fa3c1127640 10 monclient: get_auth_request con 0x7fa38c078e50 auth_method 0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.857+0000 7fa3aa7fc640 10 client.4433.objecter ms_handle_connect 0x7fa38c078e50 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.857+0000 7fa3c1127640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa38c078e50 0x7fa38c07b310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc06e9a0 tx=0x7fa3b0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.857+0000 7f0d92079640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f0d5c0813c0 0x7f0d5c083800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.857+0000 7f0d7a7fc640 10 client.4440.objecter ms_handle_connect 0x7f0d5c0813c0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa3ac01de30 con 0x7fa3bc1527f0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 10 client.4433.objecter ms_dispatch2 0x7fa3bc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 3 client.4433.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 3 client.4433.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 10 client.4433.objecter checking command 1 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7fa38c0813f0 0x7fa38c083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter _get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 15 client.4433.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter _get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 15 client.4433.objecter _session_command_op_assign 1 1 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 4 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 10 client.4433.objecter _send_command 1 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3ac00ee50 con 0x7fa38c0813f0 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 20 client.4433.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3ac05d860 con 0x7fa3bc1527f0 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3aa7fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 1 -- 10.20.193.18:0/1630791889 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c03c070 con 0x7f462c073530 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: dump: 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 1 -- 10.20.193.18:0/1630791889 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c02fb70 con 0x7f462c073530 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 1 -- 10.20.193.18:0/1630791889 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c037440 con 0x7f462c073530 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 5 monclient: authenticate success, global_id 4432 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 4 set_mon_vals no callback set 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f462a7fc640 10 monclient: dump: 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3c2129640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7fa38c0813f0 0x7fa38c083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 10 monclient: shutdown 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 -- 10.20.193.18:0/1630791889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c073530 msgr2=0x7f462c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 --2- 10.20.193.18:0/1630791889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c073530 0x7f462c15f3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f461c0098a0 tx=0x7f461c02ee60 comp rx=0 tx=0).stop 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 -- 10.20.193.18:0/1630791889 shutdown_connections 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 --2- 10.20.193.18:0/1630791889 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c073530 0x7f462c15f3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 -- 10.20.193.18:0/1630791889 >> 10.20.193.18:0/1630791889 conn(0x7f462c0ce730 msgr2=0x7f462c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7fa3c2129640 10 monclient: get_auth_request con 0x7fa38c0813f0 auth_method 0 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 -- 10.20.193.18:0/1630791889 shutdown_connections 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.858+0000 7f4631e88640 1 -- 10.20.193.18:0/1630791889 wait complete. 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 10 monclient: monmap: 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 librados: starting msgr at 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 librados: starting objecter 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 Processor -- start 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 -- start start 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 librados: setting wanted keys 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 1 librados: calling monclient init 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.859+0000 7f4631e88640 10 monclient: init 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f462c073530 0x7f462c1e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-a con 0x7f462c073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 0x7f462c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-c con 0x7f462c1527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c15f900 0x7f462c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): picked mon.noname-b con 0x7f462c15f900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0741e0 con 0x7f462c073530 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f462c073f00 con 0x7f462c15f900 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0744c0 con 0x7f462c1527f0 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.860+0000 7f4631e88640 10 monclient(hunting): authenticate will time out at 1477.541324s 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7fa3aa7fc640 10 client.4433.objecter ms_handle_connect 0x7fa38c0813f0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7fa3c2129640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7fa38c0813f0 0x7fa38c083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 0x7f462c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 0x7f462c0c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44188/0 (socket says 10.20.193.18:44188) 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 1 -- 10.20.193.18:0/2597770642 learned_addr learned my addr 10.20.193.18:0/2597770642 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 10 monclient(hunting): get_auth_request con 0x7f462c1527f0 auth_method 0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7f462affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.861+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4004480 con 0x7fecd0155eb0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: dump: 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.896 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc4015070 con 0x7fecd0155eb0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4002920 con 0x7fecd0155eb0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 5 monclient: authenticate success, global_id 4444 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 monclient: _renew_subs 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd00c6f20 con 0x7fecd0155eb0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbf7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecbdffb640 10 monclient: dump: 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 monclient: _renew_subs 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd00c74e0 con 0x7fecd0155eb0 2026-02-24T18:35:33.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 1 librados: init done 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0044070 con 0x7f66ec1527f0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: dump: 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0040610 con 0x7f66ec1527f0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e003c040 con 0x7f66ec1527f0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 5 monclient: authenticate success, global_id 4439 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 monclient: _renew_subs 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66e97fa640 4 set_mon_vals no callback set 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec0c8e70 con 0x7f66ec1527f0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66cb7fe640 10 monclient: dump: 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 client.4439.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 monclient: _renew_subs 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec0c9430 con 0x7f66ec1527f0 2026-02-24T18:35:33.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 1 librados: init done 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 client.4444.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 15 client.4444.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 20 client.4444.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fecd00d01c0 con 0x7fecd0155eb0 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f462affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 client.4439.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 15 client.4439.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 20 client.4439.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 client.4439.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.862+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66ec152bd0 con 0x7f66ec1527f0 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== osd.6 v2:10.20.193.110:6800/3466222224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa478021ec0 con 0x7fa454081390 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fa45bfff640 10 client.4424.objecter ms_dispatch2 0x7fa484000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fa45bfff640 10 client.4424.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fa45bfff640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 4 2026-02-24T18:35:33.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fa45bfff640 15 client.4424.objecter _session_command_op_remove 6 1 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd1080419e0 con 0x7fd11415fa30 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.863+0000 7fd110ff9640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0e4078e50 0x7fd0e407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.864+0000 7fd113fff640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0e4078e50 0x7fd0e407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.864+0000 7fd113fff640 10 monclient: get_auth_request con 0x7fd0e4078e50 auth_method 0 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.864+0000 7fd113fff640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0e4078e50 0x7fd0e407b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd100009750 tx=0x7fd100006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.864+0000 7fd110ff9640 10 client.4446.objecter ms_handle_connect 0x7fd0e4078e50 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fd10802fd40 con 0x7fd11415fa30 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 10 client.4446.objecter ms_dispatch2 0x7fd114000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 3 client.4446.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 3 client.4446.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 10 client.4446.objecter checking command 1 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7fd0e4081390 0x7fd0e40837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter _get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 15 client.4446.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter _get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 15 client.4446.objecter _session_command_op_assign 3 1 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 4 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 10 client.4446.objecter _send_command 1 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0e4083ea0 con 0x7fd0e4081390 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 20 client.4446.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd108083ea0 con 0x7fd11415fa30 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.867+0000 7fd110ff9640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 10 monclient(hunting): handle_auth_done global_id 4448 payload 824 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 -- 10.20.193.18:0/2597770642 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c15f900 msgr2=0x7f462c14baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f462c15f900 0x7f462c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 -- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f462c073530 msgr2=0x7f462c1e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f462c073530 0x7f462c1e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7f462affd640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1f3c70 con 0x7f462c1527f0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7fd1137fe640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7fd0e4081390 0x7fd0e40837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7fa0d19a0640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 0x7fa0cc0c8160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b7b0 tx=0x7fa0bc00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7fa0ba7fc640 10 client.?.objecter ms_handle_connect 0x7fa0cc1527f0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7fa0ba7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.868+0000 7fd1137fe640 10 monclient: get_auth_request con 0x7fd0e4081390 auth_method 0 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 10 client.4424.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 15 client.4424.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 20 client.4424.objecter _get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 20 client.4424.objecter _get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 15 client.4424.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 20 client.4424.objecter get_session s=0x7fa454080cc0 osd=6 3 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 15 client.4424.objecter _session_command_op_assign 6 2 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 4 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 10 client.4424.objecter _send_command 2 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 --> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa484152bd0 con 0x7fa454081390 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fd110ff9640 10 client.4446.objecter ms_handle_connect 0x7fd0e4081390 2026-02-24T18:35:33.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.870+0000 7fd1137fe640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7fd0e4081390 0x7fd0e40837d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.871+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3ec400d460 con 0x7f3ed015d2c0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f3ebdffb640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3ea0078e50 0x7f3ea007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f3ed5066640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3ea0078e50 0x7f3ea007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f3ed5066640 10 monclient: get_auth_request con 0x7f3ea0078e50 auth_method 0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== osd.15 v2:10.20.193.112:6802/4220209204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f0b4c083f00 con 0x7f0b4c0813f0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f0b62ffd640 10 client.4435.objecter ms_dispatch2 0x7f0b7c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f0b62ffd640 10 client.4435.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f0b62ffd640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 4 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f0b62ffd640 15 client.4435.objecter _session_command_op_remove 15 1 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f3ebdffb640 10 client.4441.objecter ms_handle_connect 0x7f3ea0078e50 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.872+0000 7f3ed5066640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3ea0078e50 0x7f3ea007b310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ed008f510 tx=0x7f3ec0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa45bfff640 1 -- 10.20.193.18:0/3856407580 <== osd.6 v2:10.20.193.110:6800/3466222224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa484152bd0 con 0x7fa454081390 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa45bfff640 10 client.4424.objecter ms_dispatch2 0x7fa484000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa45bfff640 10 client.4424.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa45bfff640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 4 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa45bfff640 15 client.4424.objecter _session_command_op_remove 6 2 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f3ec405db00 con 0x7f3ed015d2c0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 10 client.4441.objecter ms_dispatch2 0x7f3ed0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 3 client.4441.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 3 client.4441.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 10 librados: watch_flush enter 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 10 librados: watch_flush exit 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 10 client.4424.objecter close_session for osd.6 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7fa454081390 msgr2=0x7fa4540837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7fa454081390 0x7fa4540837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 20 client.4424.objecter put_session s=0x7fa454080cc0 osd=6 1 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 20 client.4424.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 10 client.4424.objecter successfully canceled tick 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 10 client.4441.objecter checking command 1 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7f3ea0081390 0x7f3ea00837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter _get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa454078e50 msgr2=0x7fa45407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa454078e50 0x7fa45407b310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa48406cdf0 tx=0x7fa46c031000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 10 monclient: shutdown 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 msgr2=0x7fa48415ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 --2- 10.20.193.18:0/3856407580 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa4841527f0 0x7fa48415ee30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa47800b780 tx=0x7fa47800bc50 comp rx=0 tx=0).stop 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4837fe640 1 -- 10.20.193.18:0/3856407580 reap_dead start 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 15 client.4441.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter _get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 15 client.4441.objecter _session_command_op_assign 14 1 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 4 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 10 client.4441.objecter _send_command 1 2026-02-24T18:35:33.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ec4018e50 con 0x7f3ea0081390 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7f3ebdffb640 20 client.4441.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 shutdown_connections 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 >> 10.20.193.18:0/3856407580 conn(0x7fa4840ce730 msgr2=0x7fa4840ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 shutdown_connections 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 -- 10.20.193.18:0/3856407580 wait complete. 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.874+0000 7fa4898c9640 1 librados: shutdown 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.875+0000 7f3ed5867640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7f3ea0081390 0x7f3ea00837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.875+0000 7f3ed5867640 10 monclient: get_auth_request con 0x7f3ea0081390 auth_method 0 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc96c03f030 con 0x7fc97c15a0f0 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7f3ed5867640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7f3ea0081390 0x7f3ea00837d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7f3ebdffb640 10 client.4441.objecter ms_handle_connect 0x7f3ea0081390 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7fc96a7fc640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc94c078e50 0x7fc94c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ec405dec0 con 0x7f3ed015d2c0 2026-02-24T18:35:33.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7f3ebdffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7fc98150c640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc94c078e50 0x7fc94c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.877+0000 7fc98150c640 10 monclient: get_auth_request con 0x7fc94c078e50 auth_method 0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.878+0000 7fc96a7fc640 10 client.4436.objecter ms_handle_connect 0x7fc94c078e50 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.878+0000 7fc98150c640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc94c078e50 0x7fc94c07b310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc970007fd0 tx=0x7fc970007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003860 con 0x7f7fc4009ab0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: dump: 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4003c70 con 0x7f7fc4009ab0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 1 -- 10.20.193.18:0/2835151145 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4005310 con 0x7f7fc4009ab0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 4 set_mon_vals no callback set 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fcf7fe640 10 monclient: dump: 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 10 monclient: shutdown 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 -- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 msgr2=0x7f7fc400bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 0x7f7fc400bea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400c320 tx=0x7f7fd402f610 comp rx=0 tx=0).stop 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 -- 10.20.193.18:0/2835151145 shutdown_connections 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 --2- 10.20.193.18:0/2835151145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fc4009ab0 0x7f7fc400bea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 -- 10.20.193.18:0/2835151145 >> 10.20.193.18:0/2835151145 conn(0x7f7fd80ce0b0 msgr2=0x7f7fd80ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 -- 10.20.193.18:0/2835151145 shutdown_connections 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f7fdede3640 1 -- 10.20.193.18:0/2835151145 wait complete. 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== osd.12 v2:10.20.193.18:6816/392025864 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f0d7c021ec0 con 0x7f0d5c0813c0 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0d7a7fc640 10 client.4440.objecter ms_dispatch2 0x7f0d8c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0d7a7fc640 10 client.4440.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0d7a7fc640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 4 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0d7a7fc640 15 client.4440.objecter _session_command_op_remove 12 1 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 10 client.4435.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 15 client.4435.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 20 client.4435.objecter _get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 20 client.4435.objecter _get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 15 client.4435.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 20 client.4435.objecter get_session s=0x7f0b4c080d20 osd=15 3 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 15 client.4435.objecter _session_command_op_assign 15 2 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 4 2026-02-24T18:35:33.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 10 client.4435.objecter _send_command 2 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.879+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 --> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b7c074ea0 con 0x7f0b4c0813f0 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient: monmap: 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 librados: starting msgr at 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 librados: starting objecter 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 Processor -- start 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 -- start start 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 librados: setting wanted keys 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 librados: calling monclient init 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient: init 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4009ab0 0x7f7fd815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-c con 0x7f7fc4009ab0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fd81ef030 0x7f7fd8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-a con 0x7f7fd81ef030 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:33.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 0x7f7fd81584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): picked mon.noname-b con 0x7f7fd81ef4b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80ccb30 con 0x7f7fd81ef030 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd815df30 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80cccb0 con 0x7f7fc4009ab0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.880+0000 7f7fdede3640 10 monclient(hunting): authenticate will time out at 1477.561853s 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fc96c083540 con 0x7fc97c15a0f0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 10 client.4436.objecter ms_dispatch2 0x7fc97c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 3 client.4436.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 3 client.4436.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 10 client.4436.objecter checking command 1 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7fc94c081390 0x7fc94c0837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter _get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 15 client.4436.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter _get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 15 client.4436.objecter _session_command_op_assign 11 1 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 4 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 10 client.4436.objecter _send_command 1 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc96c02fad0 con 0x7fc94c081390 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc96a7fc640 20 client.4436.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7f462affd640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 0x7f462c0c7ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f461c009130 tx=0x7f461c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7f4607fff640 10 client.?.objecter ms_handle_connect 0x7f462c1527f0 2026-02-24T18:35:33.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7f4607fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stdout:90194313219 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc98250e640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7fc94c081390 0x7fc94c0837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.909 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.882+0000 7fc98250e640 10 monclient: get_auth_request con 0x7fc94c081390 auth_method 0 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.884+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== osd.1 v2:10.20.193.32:6800/2795974188 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa3ac00ee50 con 0x7fa38c0813f0 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.884+0000 7fa3aa7fc640 10 client.4433.objecter ms_dispatch2 0x7fa3bc000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.884+0000 7fa3aa7fc640 10 client.4433.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.884+0000 7fa3aa7fc640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 4 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.884+0000 7fa3aa7fc640 15 client.4433.objecter _session_command_op_remove 1 1 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7fc98250e640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7fc94c081390 0x7fc94c0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7fc96a7fc640 10 client.4436.objecter ms_handle_connect 0x7fc94c081390 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff328004070 con 0x7ff32c1ee320 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7f0b62ffd640 1 -- 10.20.193.18:0/3994694550 <== osd.15 v2:10.20.193.112:6802/4220209204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b7c074ea0 con 0x7f0b4c0813f0 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7f0b62ffd640 10 client.4435.objecter ms_dispatch2 0x7f0b7c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7f0b62ffd640 10 client.4435.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7f0b62ffd640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 4 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7f0b62ffd640 15 client.4435.objecter _session_command_op_remove 15 2 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.885+0000 7ff3217fa640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2fc078e80 0x7ff2fc07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 10 librados: watch_flush enter 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 10 librados: watch_flush exit 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 10 client.4435.objecter close_session for osd.15 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f0b4c0813f0 msgr2=0x7f0b4c083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f0b4c0813f0 0x7f0b4c083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 20 client.4435.objecter put_session s=0x7f0b4c080d20 osd=15 1 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 20 client.4435.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 10 client.4435.objecter successfully canceled tick 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b4c078e50 msgr2=0x7f0b4c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b4c078e50 0x7f0b4c07b310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006fd0 tx=0x7f0b6c009190 comp rx=0 tx=0).stop 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7ff330a66640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2fc078e80 0x7ff2fc07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 10 monclient: shutdown 2026-02-24T18:35:33.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 msgr2=0x7f0b7c143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7ff330a66640 10 monclient: get_auth_request con 0x7ff2fc078e80 auth_method 0 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 --2- 10.20.193.18:0/3994694550 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b7c0d1ea0 0x7f0b7c143940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b500 tx=0x7f0b7000b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b82aa2640 1 -- 10.20.193.18:0/3994694550 reap_dead start 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 shutdown_connections 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 >> 10.20.193.18:0/3994694550 conn(0x7f0b7c072fc0 msgr2=0x7f0b7c0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 shutdown_connections 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 -- 10.20.193.18:0/3994694550 wait complete. 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0b8452c640 1 librados: shutdown 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fecc401a460 con 0x7fecd0155eb0 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 10 client.4440.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 15 client.4440.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 20 client.4440.objecter _get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 20 client.4440.objecter _get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 15 client.4440.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 20 client.4440.objecter get_session s=0x7f0d5c080cf0 osd=12 3 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 15 client.4440.objecter _session_command_op_assign 12 2 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 4 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 10 client.4440.objecter _send_command 2 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 --> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d8c0cfb80 con 0x7f0d5c0813c0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7ff330a66640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2fc078e80 0x7ff2fc07b340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009750 tx=0x7ff31c006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7ff3217fa640 10 client.4437.objecter ms_handle_connect 0x7ff2fc078e80 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7fecbdffb640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feca0078e00 0x7feca007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7fecd576c640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feca0078e00 0x7feca007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.886+0000 7fecd576c640 10 monclient: get_auth_request con 0x7feca0078e00 auth_method 0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.887+0000 7fecd576c640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feca0078e00 0x7feca007b2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecd00c2c30 tx=0x7fecc0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.887+0000 7fecbdffb640 10 client.4444.objecter ms_handle_connect 0x7feca0078e00 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fecc40978a0 con 0x7fecd0155eb0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 10 client.4444.objecter ms_dispatch2 0x7fecd0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 3 client.4444.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 3 client.4444.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 10 client.4444.objecter checking command 1 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7feca0081340 0x7feca0083780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter _get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 15 client.4444.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter _get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 15 client.4444.objecter _session_command_op_assign 9 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 4 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 10 client.4444.objecter _send_command 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecc4013e50 con 0x7feca0081340 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 20 client.4444.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fecc405e2b0 con 0x7fecd0155eb0 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecbdffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.888+0000 7fecd5f6d640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7feca0081340 0x7feca0083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fecd5f6d640 10 monclient: get_auth_request con 0x7feca0081340 auth_method 0 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== osd.3 v2:10.20.193.112:6826/230320724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd0e4083ea0 con 0x7fd0e4081390 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fd110ff9640 10 client.4446.objecter ms_dispatch2 0x7fd114000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fd110ff9640 10 client.4446.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fd110ff9640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 4 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7fd110ff9640 15 client.4446.objecter _session_command_op_remove 3 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7ff328012e00 con 0x7ff32c1ee320 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 10 client.4437.objecter ms_dispatch2 0x7ff32c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 3 client.4437.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 3 client.4437.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 10 client.4437.objecter checking command 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7ff2fc081420 0x7ff2fc083860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter _get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 15 client.4437.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter _get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 15 client.4437.objecter _session_command_op_assign 7 1 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 4 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 10 client.4437.objecter _send_command 1 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff32800ee50 con 0x7ff2fc081420 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.889+0000 7ff3217fa640 20 client.4437.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff32805e0d0 con 0x7ff32c1ee320 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7ff3217fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d7a7fc640 1 -- 10.20.193.18:0/1575185014 <== osd.12 v2:10.20.193.18:6816/392025864 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d8c0cfb80 con 0x7f0d5c0813c0 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d7a7fc640 10 client.4440.objecter ms_dispatch2 0x7f0d8c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d7a7fc640 10 client.4440.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d7a7fc640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 4 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d7a7fc640 15 client.4440.objecter _session_command_op_remove 12 2 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 10 librados: watch_flush enter 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 10 librados: watch_flush exit 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 10 client.4440.objecter close_session for osd.12 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc96c083900 con 0x7fc97c15a0f0 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7fc96a7fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f0d5c0813c0 msgr2=0x7f0d5c083800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f0d5c0813c0 0x7f0d5c083800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 20 client.4440.objecter put_session s=0x7f0d5c080cf0 osd=12 1 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 20 client.4440.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 10 client.4440.objecter successfully canceled tick 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7ff323fff640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7ff2fc081420 0x7ff2fc083860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d5c078e80 msgr2=0x7f0d5c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d5c078e80 0x7f0d5c07b340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006fd0 tx=0x7f0d80006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 10 monclient: shutdown 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 msgr2=0x7f0d8c14acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 --2- 10.20.193.18:0/1575185014 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d8c1527f0 0x7f0d8c14acc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b7b0 tx=0x7f0d7c00bc80 comp rx=0 tx=0).stop 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d92079640 1 -- 10.20.193.18:0/1575185014 reap_dead start 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 shutdown_connections 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 >> 10.20.193.18:0/1575185014 conn(0x7f0d8c0ce730 msgr2=0x7f0d8c1514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 0x7f7fd81584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 shutdown_connections 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 -- 10.20.193.18:0/1575185014 wait complete. 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f0d93b03640 1 librados: shutdown 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 0x7f7fd81584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53184/0 (socket says 10.20.193.18:53184) 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 1 -- 10.20.193.18:0/166824741 learned_addr learned my addr 10.20.193.18:0/166824741 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): get_auth_request con 0x7f7fd81ef4b0 auth_method 0 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7ff323fff640 10 monclient: get_auth_request con 0x7ff2fc081420 auth_method 0 2026-02-24T18:35:33.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5b04041830 con 0x7f5b0815fad0 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.890+0000 7f7fdd359640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7f5afd7fa640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad8078e80 0x7f5ad807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7f5b0ccca640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad8078e80 0x7f5ad807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7f5b0ccca640 10 monclient: get_auth_request con 0x7f5ad8078e80 auth_method 0 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 10 client.4433.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 15 client.4433.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 20 client.4433.objecter _get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 20 client.4433.objecter _get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 15 client.4433.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 20 client.4433.objecter get_session s=0x7fa38c080d20 osd=1 3 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 15 client.4433.objecter _session_command_op_assign 1 2 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 4 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 10 client.4433.objecter _send_command 2 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 --> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3bc152bd0 con 0x7fa38c0813f0 2026-02-24T18:35:33.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fecbdffb640 10 client.4444.objecter ms_handle_connect 0x7feca0081340 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.891+0000 7fecd5f6d640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7feca0081340 0x7feca0083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f5afd7fa640 10 client.4430.objecter ms_handle_connect 0x7f5ad8078e80 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f5b0ccca640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad8078e80 0x7f5ad807b340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5af8006fd0 tx=0x7f5af8006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0052d0 con 0x7ffbd40cccb0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: dump: 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc005490 con 0x7ffbd40cccb0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc003930 con 0x7ffbd40cccb0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbb37fe640 10 monclient: dump: 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.917 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 5 monclient: authenticate success, global_id 4455 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd17fa640 4 set_mon_vals no callback set 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 monclient: _renew_subs 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41f4e30 con 0x7ffbd40cccb0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 client.4455.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 monclient: _renew_subs 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd40c7560 con 0x7ffbd40cccb0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 1 librados: init done 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 10 monclient(hunting): handle_auth_done global_id 4447 payload 824 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 -- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4009ab0 msgr2=0x7f7fd815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7fc4009ab0 0x7f7fd815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 -- 10.20.193.18:0/166824741 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fd81ef030 msgr2=0x7f7fd8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7fd81ef030 0x7f7fd8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 monclient: found mon.noname-b 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f7fdd359640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd8158ab0 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ff323fff640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7ff2fc081420 0x7ff2fc083860 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ff3217fa640 10 client.4437.objecter ms_handle_connect 0x7ff2fc081420 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 client.4455.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 15 client.4455.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 20 client.4455.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 client.4455.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7ffbd9334640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbd40cb480 con 0x7ffbd40cccb0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.892+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f5b04083a00 con 0x7f5b0815fad0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 10 client.4430.objecter ms_dispatch2 0x7f5b08000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 3 client.4430.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 3 client.4430.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 10 client.4430.objecter checking command 1 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7f5ad80813c0 0x7f5ad8083800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter _get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 15 client.4430.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter _get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 15 client.4430.objecter _session_command_op_assign 5 1 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 4 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 10 client.4430.objecter _send_command 1 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ad8083ed0 con 0x7f5ad80813c0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 20 client.4430.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b04083dc0 con 0x7f5b0815fad0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afd7fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afffff640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7f5ad80813c0 0x7f5ad8083800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f7fdd359640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 0x7f7fd81584b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40059b0 tx=0x7f7fd4002e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f7fcd7fa640 10 client.?.objecter ms_handle_connect 0x7f7fd81ef4b0 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f7fcd7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:33.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.893+0000 7f5afffff640 10 monclient: get_auth_request con 0x7f5ad80813c0 auth_method 0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc004480 con 0x7fa0cc1527f0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: dump: 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc010070 con 0x7fa0cc1527f0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc002920 con 0x7fa0cc1527f0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0ba7fc640 10 monclient: dump: 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0bbfff640 4 set_mon_vals no callback set 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 5 monclient: authenticate success, global_id 4442 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 monclient: _renew_subs 2026-02-24T18:35:33.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc0c70c0 con 0x7fa0cc1527f0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3aa7fc640 1 -- 10.20.193.18:0/1779742788 <== osd.1 v2:10.20.193.32:6800/2795974188 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3bc152bd0 con 0x7fa38c0813f0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3aa7fc640 10 client.4433.objecter ms_dispatch2 0x7fa3bc000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3aa7fc640 10 client.4433.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3aa7fc640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 4 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3aa7fc640 15 client.4433.objecter _session_command_op_remove 1 2 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 client.4442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 monclient: _renew_subs 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc0c7680 con 0x7fa0cc1527f0 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 1 librados: init done 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 10 librados: watch_flush enter 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 10 librados: watch_flush exit 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 10 client.4433.objecter close_session for osd.1 2026-02-24T18:35:33.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7fa38c0813f0 msgr2=0x7fa38c083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7fa38c0813f0 0x7fa38c083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 20 client.4433.objecter put_session s=0x7fa38c080d20 osd=1 1 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 20 client.4433.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 10 client.4433.objecter successfully canceled tick 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa38c078e50 msgr2=0x7fa38c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa38c078e50 0x7fa38c07b310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc06e9a0 tx=0x7fa3b0031000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 10 monclient: shutdown 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 msgr2=0x7fa3bc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 --2- 10.20.193.18:0/1779742788 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa3bc1527f0 0x7fa3bc0c7db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b780 tx=0x7fa3ac00bc50 comp rx=0 tx=0).stop 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c2129640 1 -- 10.20.193.18:0/1779742788 reap_dead start 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 shutdown_connections 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 >> 10.20.193.18:0/1779742788 conn(0x7fa3bc0ce730 msgr2=0x7fa3bc0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 shutdown_connections 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 -- 10.20.193.18:0/1779742788 wait complete. 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa3c3bb3640 1 librados: shutdown 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 client.4442.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 15 client.4442.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 20 client.4442.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 client.4442.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0cc076e00 con 0x7fa0cc1527f0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7f5afffff640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7f5ad80813c0 0x7f5ad8083800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.895+0000 7f5afd7fa640 10 client.4430.objecter ms_handle_connect 0x7f5ad80813c0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003860 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: dump: 2026-02-24T18:35:33.922 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4003e20 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 10 client.4446.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 15 client.4446.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 20 client.4446.objecter _get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4036850 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fceffd640 4 set_mon_vals no callback set 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fcd7fa640 10 monclient: dump: 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 5 monclient: authenticate success, global_id 4447 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 10 monclient: _renew_subs 2026-02-24T18:35:33.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 20 client.4446.objecter _get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 15 client.4446.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 20 client.4446.objecter get_session s=0x7fd0e4080cc0 osd=3 3 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 15 client.4446.objecter _session_command_op_assign 3 2 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 4 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 10 client.4446.objecter _send_command 2 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd8158d90 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1140cc020 con 0x7fd0e4081390 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 10 client.4447.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 10 monclient: _renew_subs 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd8159350 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.896+0000 7f7fdede3640 1 librados: init done 2026-02-24T18:35:33.924 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:33.924 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 10 client.4447.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 15 client.4447.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 20 client.4447.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 10 client.4447.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.897+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fd8076a80 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:33.925 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd110ff9640 1 -- 10.20.193.18:0/3318530097 <== osd.3 v2:10.20.193.112:6826/230320724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd1140cc020 con 0x7fd0e4081390 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd110ff9640 10 client.4446.objecter ms_dispatch2 0x7fd114000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd110ff9640 10 client.4446.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd110ff9640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 4 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd110ff9640 15 client.4446.objecter _session_command_op_remove 3 2 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 10 librados: watch_flush enter 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 10 librados: watch_flush exit 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 10 client.4446.objecter close_session for osd.3 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7fd0e4081390 msgr2=0x7fd0e40837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7fd0e4081390 0x7fd0e40837d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 20 client.4446.objecter put_session s=0x7fd0e4080cc0 osd=3 1 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 20 client.4446.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 10 client.4446.objecter successfully canceled tick 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0e4078e50 msgr2=0x7fd0e407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0e4078e50 0x7fd0e407b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd100009750 tx=0x7fd100006ca0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 10 monclient: shutdown 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 msgr2=0x7fd11414a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 --2- 10.20.193.18:0/3318530097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd11415fa30 0x7fd11414a280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd10802f6f0 tx=0x7fd108004bb0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd118a60640 1 -- 10.20.193.18:0/3318530097 reap_dead start 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 shutdown_connections 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.900+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 >> 10.20.193.18:0/3318530097 conn(0x7fd114072fc0 msgr2=0x7fd1140cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.901+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 shutdown_connections 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.901+0000 7fd11a4ea640 1 -- 10.20.193.18:0/3318530097 wait complete. 2026-02-24T18:35:33.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.901+0000 7fd11a4ea640 1 librados: shutdown 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.902+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f66e003f050 con 0x7f66ec1527f0 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.903+0000 7f66cb7fe640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f66b8078e50 0x7f66b807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.903+0000 7f66ea7fc640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f66b8078e50 0x7f66b807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.903+0000 7f66ea7fc640 10 monclient: get_auth_request con 0x7f66b8078e50 auth_method 0 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.904+0000 7f66cb7fe640 10 client.4439.objecter ms_handle_connect 0x7f66b8078e50 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.904+0000 7f66ea7fc640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f66b8078e50 0x7f66b807b310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66ec1517f0 tx=0x7f66d4009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.927 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:33.927 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f66e0082f30 con 0x7f66ec1527f0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 10 client.4439.objecter ms_dispatch2 0x7f66ec000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 3 client.4439.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 3 client.4439.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 10 client.4439.objecter checking command 1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f66b80813f0 0x7f66b8083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter _get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 15 client.4439.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter _get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 15 client.4439.objecter _session_command_op_assign 10 1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 4 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 10 client.4439.objecter _send_command 1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66e002fb40 con 0x7f66b80813f0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 20 client.4439.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66e00832f0 con 0x7f66ec1527f0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66cb7fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66eb7fe640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f66b80813f0 0x7f66b8083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.905+0000 7f66eb7fe640 10 monclient: get_auth_request con 0x7f66b80813f0 auth_method 0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== osd.11 v2:10.20.193.112:6812/2071089960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fc96c02fad0 con 0x7fc94c081390 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7fc96a7fc640 10 client.4436.objecter ms_dispatch2 0x7fc97c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7fc96a7fc640 10 client.4436.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7fc96a7fc640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 4 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7fc96a7fc640 15 client.4436.objecter _session_command_op_remove 11 1 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.908+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ffbbc003ad0 con 0x7ffbd40cccb0 2026-02-24T18:35:33.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7ffbb37fe640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffba0078e50 0x7ffba007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:33.929 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7ffbd2ffd640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffba0078e50 0x7ffba007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7ffbd2ffd640 10 monclient: get_auth_request con 0x7ffba0078e50 auth_method 0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7fd40369f0 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7f66cb7fe640 10 client.4439.objecter ms_handle_connect 0x7f66b80813f0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7f66eb7fe640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f66b80813f0 0x7f66b8083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7ffbd2ffd640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffba0078e50 0x7ffba007b310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005d00 tx=0x7ffbc4002e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7ffbb37fe640 10 client.4455.objecter ms_handle_connect 0x7ffba0078e50 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.909+0000 7f7fcd7fa640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7fa8078e80 0x7f7fa807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f7fdcb58640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7fa8078e80 0x7f7fa807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f7fdcb58640 10 monclient: get_auth_request con 0x7f7fa8078e80 auth_method 0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c03c040 con 0x7f462c1527f0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: dump: 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c042630 con 0x7f462c1527f0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c042a20 con 0x7f462c1527f0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4607fff640 10 monclient: dump: 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.930 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4629ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 5 monclient: authenticate success, global_id 4448 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 monclient: _renew_subs 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1f3f50 con 0x7f462c1527f0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 client.4448.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 monclient: _renew_subs 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1f4510 con 0x7f462c1527f0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 1 librados: init done 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f7fdcb58640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7fa8078e80 0x7f7fa807b340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8002a30 tx=0x7f7fc8003210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f7fcd7fa640 10 client.4447.objecter ms_handle_connect 0x7f7fa8078e80 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 client.4448.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 15 client.4448.objecter _session_command_op_assign -1 1 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 20 client.4448.objecter _get_session osd=-1 returning homeless 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 client.4448.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.910+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f462c0747a0 con 0x7f462c1527f0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f7fd4036d10 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 10 client.4447.objecter ms_dispatch2 0x7f7fd8000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 3 client.4447.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 3 client.4447.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 10 client.4447.objecter checking command 1 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7f7fa80813c0 0x7f7fa8083800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter _get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 15 client.4447.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter _get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 15 client.4447.objecter _session_command_op_assign 2 1 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 4 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 10 client.4447.objecter _send_command 1 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fd4041e50 con 0x7f7fa80813c0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 20 client.4447.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fd4082b80 con 0x7f7fd81ef4b0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.911+0000 7f7fcd7fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7f7fcffff640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7f7fa80813c0 0x7f7fa8083800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7f7fcffff640 10 monclient: get_auth_request con 0x7f7fa80813c0 auth_method 0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== osd.7 v2:10.20.193.112:6810/92089351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ff32800ee50 con 0x7ff2fc081420 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7ff3217fa640 10 client.4437.objecter ms_dispatch2 0x7ff32c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7ff3217fa640 10 client.4437.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7ff3217fa640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 4 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7ff3217fa640 15 client.4437.objecter _session_command_op_remove 7 1 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== osd.9 v2:10.20.193.32:6816/1806399530 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fecc4013e50 con 0x7feca0081340 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7fecbdffb640 10 client.4444.objecter ms_dispatch2 0x7fecd0000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7fecbdffb640 10 client.4444.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7fecbdffb640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 4 2026-02-24T18:35:33.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.912+0000 7fecbdffb640 15 client.4444.objecter _session_command_op_remove 9 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7ffbbc023db0 con 0x7ffbd40cccb0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 10 client.4455.objecter ms_dispatch2 0x7ffbd4000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 3 client.4455.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 3 client.4455.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 10 client.4455.objecter checking command 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7ffba00813f0 0x7ffba0083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter _get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 15 client.4455.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter _get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 15 client.4455.objecter _session_command_op_assign 13 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 4 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 10 client.4455.objecter _send_command 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbbc024ec0 con 0x7ffba00813f0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 20 client.4455.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc05e330 con 0x7ffbd40cccb0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbb37fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbd27fc640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7ffba00813f0 0x7ffba0083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.913+0000 7ffbd27fc640 10 monclient: get_auth_request con 0x7ffba00813f0 auth_method 0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== osd.14 v2:10.20.193.110:6816/3577843273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3ec4018e50 con 0x7f3ea0081390 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f3ebdffb640 10 client.4441.objecter ms_dispatch2 0x7f3ed0000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f3ebdffb640 10 client.4441.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f3ebdffb640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 4 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f3ebdffb640 15 client.4441.objecter _session_command_op_remove 14 1 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f7fcffff640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7f7fa80813c0 0x7f7fa8083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.914+0000 7f7fcd7fa640 10 client.4447.objecter ms_handle_connect 0x7f7fa80813c0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 10 client.4436.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 15 client.4436.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 20 client.4436.objecter _get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 20 client.4436.objecter _get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 15 client.4436.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 20 client.4436.objecter get_session s=0x7fc94c080cc0 osd=11 3 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 15 client.4436.objecter _session_command_op_assign 11 2 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 4 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 10 client.4436.objecter _send_command 2 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.915+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 --> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc97c0c4a90 con 0x7fc94c081390 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.916+0000 7ffbd27fc640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7ffba00813f0 0x7ffba0083830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.916+0000 7ffbb37fe640 10 client.4455.objecter ms_handle_connect 0x7ffba00813f0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa0bc01d030 con 0x7fa0cc1527f0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== osd.5 v2:10.20.193.32:6808/2128834550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5ad8083ed0 con 0x7f5ad80813c0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7f5afd7fa640 10 client.4430.objecter ms_dispatch2 0x7f5b08000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7f5afd7fa640 10 client.4430.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7f5afd7fa640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 4 2026-02-24T18:35:33.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7f5afd7fa640 15 client.4430.objecter _session_command_op_remove 5 1 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7fa0ba7fc640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa09c078e50 0x7fa09c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7fa0d119f640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa09c078e50 0x7fa09c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.918+0000 7fa0d119f640 10 monclient: get_auth_request con 0x7fa09c078e50 auth_method 0 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fa0ba7fc640 10 client.4442.objecter ms_handle_connect 0x7fa09c078e50 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fa0d119f640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa09c078e50 0x7fa09c07b310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0039000 tx=0x7fa0c0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 10 client.4444.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 15 client.4444.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 20 client.4444.objecter _get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 20 client.4444.objecter _get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 15 client.4444.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 20 client.4444.objecter get_session s=0x7feca0080c70 osd=9 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 15 client.4444.objecter _session_command_op_assign 9 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 4 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 10 client.4444.objecter _send_command 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 --> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fecd01596d0 con 0x7feca0081340 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 10 client.4437.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 15 client.4437.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 20 client.4437.objecter _get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 20 client.4437.objecter _get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 15 client.4437.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 20 client.4437.objecter get_session s=0x7ff2fc080d50 osd=7 3 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 15 client.4437.objecter _session_command_op_assign 7 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 4 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 10 client.4437.objecter _send_command 2 2026-02-24T18:35:33.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.919+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 --> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff32c163510 con 0x7ff2fc081420 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa0bc05d520 con 0x7fa0cc1527f0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 10 client.4442.objecter ms_dispatch2 0x7fa0cc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 3 client.4442.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 3 client.4442.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 10 client.4442.objecter checking command 1 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7fa09c0813f0 0x7fa09c083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter _get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 15 client.4442.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter _get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 15 client.4442.objecter _session_command_op_assign 4 1 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 4 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 10 client.4442.objecter _send_command 1 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0bc022ec0 con 0x7fa09c0813f0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 20 client.4442.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0d21a1640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7fa09c0813f0 0x7fa09c083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc05d8e0 con 0x7fa0cc1527f0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0ba7fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.920+0000 7fa0d21a1640 10 monclient: get_auth_request con 0x7fa09c0813f0 auth_method 0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7fa0ba7fc640 10 client.4442.objecter ms_handle_connect 0x7fa09c0813f0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7fa0d21a1640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7fa09c0813f0 0x7fa09c083830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 10 client.4441.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 15 client.4441.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 20 client.4441.objecter _get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 20 client.4441.objecter _get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 15 client.4441.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 20 client.4441.objecter get_session s=0x7f3ea0080cc0 osd=14 3 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 15 client.4441.objecter _session_command_op_assign 14 2 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 4 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 10 client.4441.objecter _send_command 2 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 --> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ed015cce0 con 0x7f3ea0081390 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff3217fa640 1 -- 10.20.193.18:0/2144418042 <== osd.7 v2:10.20.193.112:6810/92089351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff32c163510 con 0x7ff2fc081420 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff3217fa640 10 client.4437.objecter ms_dispatch2 0x7ff32c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff3217fa640 10 client.4437.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff3217fa640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 4 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff3217fa640 15 client.4437.objecter _session_command_op_remove 7 2 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 10 librados: watch_flush enter 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 10 librados: watch_flush exit 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 10 client.4437.objecter close_session for osd.7 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7ff2fc081420 msgr2=0x7ff2fc083860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7ff2fc081420 0x7ff2fc083860 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 20 client.4437.objecter put_session s=0x7ff2fc080d50 osd=7 1 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 20 client.4437.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.922+0000 7ff332cf1640 10 client.4437.objecter successfully canceled tick 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2fc078e80 msgr2=0x7ff2fc07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2fc078e80 0x7ff2fc07b340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009750 tx=0x7ff31c006ca0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 10 monclient: shutdown 2026-02-24T18:35:33.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 msgr2=0x7ff32c158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 --2- 10.20.193.18:0/2144418042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff32c1ee320 0x7ff32c158df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b9a0 tx=0x7ff32800be70 comp rx=0 tx=0).stop 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff331267640 1 -- 10.20.193.18:0/2144418042 reap_dead start 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 shutdown_connections 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 >> 10.20.193.18:0/2144418042 conn(0x7ff32c0ca1f0 msgr2=0x7ff32c0740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 shutdown_connections 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 -- 10.20.193.18:0/2144418042 wait complete. 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7ff332cf1640 1 librados: shutdown 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecbdffb640 1 -- 10.20.193.18:0/2719163345 <== osd.9 v2:10.20.193.32:6816/1806399530 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fecd01596d0 con 0x7feca0081340 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecbdffb640 10 client.4444.objecter ms_dispatch2 0x7fecd0000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecbdffb640 10 client.4444.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecbdffb640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 4 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecbdffb640 15 client.4444.objecter _session_command_op_remove 9 2 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 10 librados: watch_flush enter 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 10 librados: watch_flush exit 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 10 client.4444.objecter close_session for osd.9 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7feca0081340 msgr2=0x7feca0083780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7feca0081340 0x7feca0083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 20 client.4444.objecter put_session s=0x7feca0080c70 osd=9 1 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 20 client.4444.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 10 client.4444.objecter successfully canceled tick 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feca0078e00 msgr2=0x7feca007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7feca0078e00 0x7feca007b2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecd00c2c30 tx=0x7fecc0033000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 10 monclient: shutdown 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 msgr2=0x7fecd00c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 --2- 10.20.193.18:0/2719163345 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fecd0155eb0 0x7fecd00c7fc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecc400b470 tx=0x7fecc400b940 comp rx=0 tx=0).stop 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd5f6d640 1 -- 10.20.193.18:0/2719163345 reap_dead start 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 shutdown_connections 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 >> 10.20.193.18:0/2719163345 conn(0x7fecd00cf580 msgr2=0x7fecd015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 shutdown_connections 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 -- 10.20.193.18:0/2719163345 wait complete. 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fecd79f7640 1 librados: shutdown 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fc96a7fc640 1 -- 10.20.193.18:0/2674794266 <== osd.11 v2:10.20.193.112:6812/2071089960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc97c0c4a90 con 0x7fc94c081390 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fc96a7fc640 10 client.4436.objecter ms_dispatch2 0x7fc97c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fc96a7fc640 10 client.4436.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fc96a7fc640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 4 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.923+0000 7fc96a7fc640 15 client.4436.objecter _session_command_op_remove 11 2 2026-02-24T18:35:33.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 10 librados: watch_flush enter 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 10 librados: watch_flush exit 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 10 client.4436.objecter close_session for osd.11 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7fc94c081390 msgr2=0x7fc94c0837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7fc94c081390 0x7fc94c0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 20 client.4436.objecter put_session s=0x7fc94c080cc0 osd=11 1 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 20 client.4436.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 10 client.4436.objecter successfully canceled tick 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc94c078e50 msgr2=0x7fc94c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc94c078e50 0x7fc94c07b310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc970007fd0 tx=0x7fc970007b40 comp rx=0 tx=0).stop 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 10 monclient: shutdown 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 msgr2=0x7fc97c1fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 --2- 10.20.193.18:0/2674794266 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc97c15a0f0 0x7fc97c1fc180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0099d0 tx=0x7fc96c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc98250e640 1 -- 10.20.193.18:0/2674794266 reap_dead start 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 shutdown_connections 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 >> 10.20.193.18:0/2674794266 conn(0x7fc97c06d8c0 msgr2=0x7fc97c158550 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 shutdown_connections 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 -- 10.20.193.18:0/2674794266 wait complete. 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.924+0000 7fc983f98640 1 librados: shutdown 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 10 client.4430.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 15 client.4430.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 20 client.4430.objecter _get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 20 client.4430.objecter _get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 15 client.4430.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 20 client.4430.objecter get_session s=0x7f5ad8080cf0 osd=5 3 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 15 client.4430.objecter _session_command_op_assign 5 2 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 4 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 10 client.4430.objecter _send_command 2 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.925+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 --> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b081561e0 con 0x7f5ad80813c0 2026-02-24T18:35:33.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ebdffb640 1 -- 10.20.193.18:0/3335183108 <== osd.14 v2:10.20.193.110:6816/3577843273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ed015cce0 con 0x7f3ea0081390 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ebdffb640 10 client.4441.objecter ms_dispatch2 0x7f3ed0000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ebdffb640 10 client.4441.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ebdffb640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 4 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ebdffb640 15 client.4441.objecter _session_command_op_remove 14 2 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5afd7fa640 1 -- 10.20.193.18:0/161218493 <== osd.5 v2:10.20.193.32:6808/2128834550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b081561e0 con 0x7f5ad80813c0 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5afd7fa640 10 client.4430.objecter ms_dispatch2 0x7f5b08000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5afd7fa640 10 client.4430.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5afd7fa640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 4 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5afd7fa640 15 client.4430.objecter _session_command_op_remove 5 2 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 10 librados: watch_flush enter 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 10 librados: watch_flush exit 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 10 client.4430.objecter close_session for osd.5 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7f5ad80813c0 msgr2=0x7f5ad8083800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7f5ad80813c0 0x7f5ad8083800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 20 client.4430.objecter put_session s=0x7f5ad8080cf0 osd=5 1 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 20 client.4430.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f5b0ef55640 10 client.4430.objecter successfully canceled tick 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad8078e80 msgr2=0x7f5ad807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5ad8078e80 0x7f5ad807b340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5af8006fd0 tx=0x7f5af8006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 10 monclient: shutdown 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 msgr2=0x7f5b081f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 --2- 10.20.193.18:0/161218493 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5b0815fad0 0x7f5b081f4d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b04039040 tx=0x7f5b04004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0d4cb640 1 -- 10.20.193.18:0/161218493 reap_dead start 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 shutdown_connections 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 >> 10.20.193.18:0/161218493 conn(0x7f5b080ce730 msgr2=0x7f5b080ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 shutdown_connections 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 -- 10.20.193.18:0/161218493 wait complete. 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f5b0ef55640 1 librados: shutdown 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 10 librados: watch_flush enter 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 10 librados: watch_flush exit 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 10 client.4441.objecter close_session for osd.14 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7f3ea0081390 msgr2=0x7f3ea00837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7f3ea0081390 0x7f3ea00837d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 20 client.4441.objecter put_session s=0x7f3ea0080cc0 osd=14 1 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 20 client.4441.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 10 client.4441.objecter successfully canceled tick 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3ea0078e50 msgr2=0x7f3ea007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3ea0078e50 0x7f3ea007b310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ed008f510 tx=0x7f3ec0031000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 10 monclient: shutdown 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 msgr2=0x7f3ed014c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed72f1640 1 --2- 10.20.193.18:0/3335183108 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3ed015d2c0 0x7f3ed014c240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b5a0 tx=0x7f3ec400ba70 comp rx=0 tx=0).stop 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.928+0000 7f3ed5867640 1 -- 10.20.193.18:0/3335183108 reap_dead start 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 shutdown_connections 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 >> 10.20.193.18:0/3335183108 conn(0x7f3ed00ce730 msgr2=0x7f3ed006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 shutdown_connections 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f3ed72f1640 1 -- 10.20.193.18:0/3335183108 wait complete. 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.929+0000 7f3ed72f1640 1 librados: shutdown 2026-02-24T18:35:33.942 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:33.943 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-24T18:35:33.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.932+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f461c04b450 con 0x7f462c1527f0 2026-02-24T18:35:33.943 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:33.943 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-24T18:35:33.943 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:33.944 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f4607fff640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f45fc078e50 0x7f45fc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== osd.10 v2:10.20.193.110:6808/4149932150 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f66e002fb40 con 0x7f66b80813f0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f66cb7fe640 10 client.4439.objecter ms_dispatch2 0x7f66ec000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f66cb7fe640 10 client.4439.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f66cb7fe640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 4 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f66cb7fe640 15 client.4439.objecter _session_command_op_remove 10 1 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f462b7fe640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f45fc078e50 0x7f45fc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.933+0000 7f462b7fe640 10 monclient: get_auth_request con 0x7f45fc078e50 auth_method 0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.934+0000 7f462b7fe640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f45fc078e50 0x7f45fc07b310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4618002f00 tx=0x7f4618007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.934+0000 7f4607fff640 10 client.4448.objecter ms_handle_connect 0x7f45fc078e50 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f461c02fab0 con 0x7f462c1527f0 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 10 client.4448.objecter ms_dispatch2 0x7f462c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:33.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 3 client.4448.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 3 client.4448.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 10 client.4448.objecter checking command 1 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7f45fc0813f0 0x7f45fc083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter _get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 15 client.4448.objecter _session_command_op_remove -1 1 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter _get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 15 client.4448.objecter _session_command_op_assign 8 1 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 4 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 10 client.4448.objecter _send_command 1 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f461c004590 con 0x7f45fc0813f0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 20 client.4448.objecter dump_active .. 0 homeless 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f461c082db0 con 0x7f462c1527f0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f4607fff640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f462bfff640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7f45fc0813f0 0x7f45fc083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.935+0000 7f462bfff640 10 monclient: get_auth_request con 0x7f45fc0813f0 auth_method 0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== osd.2 v2:10.20.193.110:6824/3650974883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f7fd4041e50 con 0x7f7fa80813c0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f7fcd7fa640 10 client.4447.objecter ms_dispatch2 0x7f7fd8000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f7fcd7fa640 10 client.4447.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f7fcd7fa640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 4 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f7fcd7fa640 15 client.4447.objecter _session_command_op_remove 2 1 2026-02-24T18:35:33.945 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:33.945 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f4607fff640 10 client.4448.objecter ms_handle_connect 0x7f45fc0813f0 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.937+0000 7f462bfff640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7f45fc0813f0 0x7f45fc083830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:33.946 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 10 client.4439.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 15 client.4439.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 20 client.4439.objecter _get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 20 client.4439.objecter _get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 15 client.4439.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 20 client.4439.objecter get_session s=0x7f66b8080d20 osd=10 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 15 client.4439.objecter _session_command_op_assign 10 2 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 4 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 10 client.4439.objecter _send_command 2 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.940+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 --> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f66ec06bd70 con 0x7f66b80813f0 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 10 client.4447.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 15 client.4447.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 20 client.4447.objecter _get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 20 client.4447.objecter _get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 15 client.4447.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 20 client.4447.objecter get_session s=0x7f7fa8080cf0 osd=2 3 2026-02-24T18:35:33.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 15 client.4447.objecter _session_command_op_assign 2 2 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 4 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 10 client.4447.objecter _send_command 2 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.944+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 --> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fd80cbbd0 con 0x7f7fa80813c0 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66cb7fe640 1 -- 10.20.193.18:0/3469857683 <== osd.10 v2:10.20.193.110:6808/4149932150 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f66ec06bd70 con 0x7f66b80813f0 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66cb7fe640 10 client.4439.objecter ms_dispatch2 0x7f66ec000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66cb7fe640 10 client.4439.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66cb7fe640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 4 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66cb7fe640 15 client.4439.objecter _session_command_op_remove 10 2 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 10 librados: watch_flush enter 2026-02-24T18:35:33.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 10 librados: watch_flush exit 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 10 client.4439.objecter close_session for osd.10 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f66b80813f0 msgr2=0x7f66b8083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f66b80813f0 0x7f66b8083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 20 client.4439.objecter put_session s=0x7f66b8080d20 osd=10 1 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 20 client.4439.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 10 client.4439.objecter successfully canceled tick 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f66b8078e50 msgr2=0x7f66b807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f66b8078e50 0x7f66b807b310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66ec1517f0 tx=0x7f66d4009290 comp rx=0 tx=0).stop 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 10 monclient: shutdown 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66ec1527f0 msgr2=0x7f66ec0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 --2- 10.20.193.18:0/3469857683 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66ec1527f0 0x7f66ec0ca1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e00047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66eb7fe640 1 -- 10.20.193.18:0/3469857683 reap_dead start 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 shutdown_connections 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 >> 10.20.193.18:0/3469857683 conn(0x7f66ec0ce730 msgr2=0x7f66ec0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 shutdown_connections 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 -- 10.20.193.18:0/3469857683 wait complete. 2026-02-24T18:35:33.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.946+0000 7f66f16f5640 1 librados: shutdown 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fcd7fa640 1 -- 10.20.193.18:0/166824741 <== osd.2 v2:10.20.193.110:6824/3650974883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7fd80cbbd0 con 0x7f7fa80813c0 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fcd7fa640 10 client.4447.objecter ms_dispatch2 0x7f7fd8000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fcd7fa640 10 client.4447.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fcd7fa640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 4 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fcd7fa640 15 client.4447.objecter _session_command_op_remove 2 2 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 10 librados: watch_flush enter 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 10 librados: watch_flush exit 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 10 client.4447.objecter close_session for osd.2 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7f7fa80813c0 msgr2=0x7f7fa8083800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7f7fa80813c0 0x7f7fa8083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 20 client.4447.objecter put_session s=0x7f7fa8080cf0 osd=2 1 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 20 client.4447.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 10 client.4447.objecter successfully canceled tick 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7fa8078e80 msgr2=0x7f7fa807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7fa8078e80 0x7f7fa807b340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8002a30 tx=0x7f7fc8003210 comp rx=0 tx=0).stop 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 10 monclient: shutdown 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 msgr2=0x7f7fd81584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 --2- 10.20.193.18:0/166824741 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7fd81ef4b0 0x7f7fd81584b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40059b0 tx=0x7f7fd4002e40 comp rx=0 tx=0).stop 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdd359640 1 -- 10.20.193.18:0/166824741 reap_dead start 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 shutdown_connections 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 >> 10.20.193.18:0/166824741 conn(0x7f7fd80ce0b0 msgr2=0x7f7fd81529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 shutdown_connections 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.950+0000 7f7fdede3640 1 -- 10.20.193.18:0/166824741 wait complete. 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.951+0000 7f7fdede3640 1 librados: shutdown 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.948+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== osd.4 v2:10.20.193.18:6800/3481454300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa0bc022ec0 con 0x7fa09c0813f0 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.948+0000 7fa0ba7fc640 10 client.4442.objecter ms_dispatch2 0x7fa0cc000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.948+0000 7fa0ba7fc640 10 client.4442.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.948+0000 7fa0ba7fc640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 4 2026-02-24T18:35:33.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.948+0000 7fa0ba7fc640 15 client.4442.objecter _session_command_op_remove 4 1 2026-02-24T18:35:33.957 INFO:teuthology.orchestra.run.trial018.stdout:90194313220 2026-02-24T18:35:33.957 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 10 client.4442.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 15 client.4442.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 20 client.4442.objecter _get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 20 client.4442.objecter _get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 15 client.4442.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 20 client.4442.objecter get_session s=0x7fa09c080d20 osd=4 3 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 15 client.4442.objecter _session_command_op_assign 4 2 2026-02-24T18:35:33.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 4 2026-02-24T18:35:33.959 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 10 client.4442.objecter _send_command 2 2026-02-24T18:35:33.959 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.957+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 --> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0cc0d05b0 con 0x7fa09c0813f0 2026-02-24T18:35:33.961 INFO:teuthology.orchestra.run.trial018.stdout:90194313220 2026-02-24T18:35:33.961 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:35:33.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0ba7fc640 1 -- 10.20.193.18:0/1784390635 <== osd.4 v2:10.20.193.18:6800/3481454300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0cc0d05b0 con 0x7fa09c0813f0 2026-02-24T18:35:33.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0ba7fc640 10 client.4442.objecter ms_dispatch2 0x7fa0cc000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0ba7fc640 10 client.4442.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0ba7fc640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 4 2026-02-24T18:35:33.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0ba7fc640 15 client.4442.objecter _session_command_op_remove 4 2 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 10 librados: watch_flush enter 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 10 librados: watch_flush exit 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 10 client.4442.objecter close_session for osd.4 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7fa09c0813f0 msgr2=0x7fa09c083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7fa09c0813f0 0x7fa09c083830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 20 client.4442.objecter put_session s=0x7fa09c080d20 osd=4 1 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 20 client.4442.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 10 client.4442.objecter successfully canceled tick 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa09c078e50 msgr2=0x7fa09c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa09c078e50 0x7fa09c07b310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0039000 tx=0x7fa0c0031000 comp rx=0 tx=0).stop 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 10 monclient: shutdown 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 msgr2=0x7fa0cc0c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d3c2b640 1 --2- 10.20.193.18:0/1784390635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0cc1527f0 0x7fa0cc0c8160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b7b0 tx=0x7fa0bc00bc80 comp rx=0 tx=0).stop 2026-02-24T18:35:33.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.962+0000 7fa0d21a1640 1 -- 10.20.193.18:0/1784390635 reap_dead start 2026-02-24T18:35:33.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.963+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 shutdown_connections 2026-02-24T18:35:33.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.963+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 >> 10.20.193.18:0/1784390635 conn(0x7fa0cc0ce730 msgr2=0x7fa0cc0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.963+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 shutdown_connections 2026-02-24T18:35:33.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.963+0000 7fa0d3c2b640 1 -- 10.20.193.18:0/1784390635 wait complete. 2026-02-24T18:35:33.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.963+0000 7fa0d3c2b640 1 librados: shutdown 2026-02-24T18:35:33.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.965+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== osd.8 v2:10.20.193.18:6808/1977979333 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f461c004590 con 0x7f45fc0813f0 2026-02-24T18:35:33.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.965+0000 7f4607fff640 10 client.4448.objecter ms_dispatch2 0x7f462c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:33.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.965+0000 7f4607fff640 10 client.4448.objecter _finish_command 1 = system:0 2026-02-24T18:35:33.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.965+0000 7f4607fff640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 4 2026-02-24T18:35:33.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.965+0000 7f4607fff640 15 client.4448.objecter _session_command_op_remove 8 1 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:33.975 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 10 client.4448.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 15 client.4448.objecter _session_command_op_assign -1 2 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 20 client.4448.objecter _get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 20 client.4448.objecter _get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 15 client.4448.objecter _session_command_op_remove -1 2 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 20 client.4448.objecter get_session s=0x7f45fc080d20 osd=8 3 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 15 client.4448.objecter _session_command_op_assign 8 2 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 4 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 10 client.4448.objecter _send_command 2 2026-02-24T18:35:33.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.974+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 --> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f462c1e75c0 con 0x7f45fc0813f0 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4607fff640 1 -- 10.20.193.18:0/2597770642 <== osd.8 v2:10.20.193.18:6808/1977979333 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f462c1e75c0 con 0x7f45fc0813f0 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4607fff640 10 client.4448.objecter ms_dispatch2 0x7f462c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4607fff640 10 client.4448.objecter _finish_command 2 = system:0 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4607fff640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 4 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4607fff640 15 client.4448.objecter _session_command_op_remove 8 2 2026-02-24T18:35:33.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 10 librados: watch_flush enter 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 10 librados: watch_flush exit 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 10 client.4448.objecter close_session for osd.8 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7f45fc0813f0 msgr2=0x7f45fc083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7f45fc0813f0 0x7f45fc083830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 20 client.4448.objecter put_session s=0x7f45fc080d20 osd=8 1 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 20 client.4448.objecter shutdown clearing up homeless session... 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 10 client.4448.objecter successfully canceled tick 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f45fc078e50 msgr2=0x7f45fc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f45fc078e50 0x7f45fc07b310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4618002f00 tx=0x7f4618007a70 comp rx=0 tx=0).stop 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 10 monclient: shutdown 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 msgr2=0x7f462c0c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f4631e88640 1 --2- 10.20.193.18:0/2597770642 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f462c1527f0 0x7f462c0c7ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f461c009130 tx=0x7f461c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.979+0000 7f462bfff640 1 -- 10.20.193.18:0/2597770642 reap_dead start 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.980+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 shutdown_connections 2026-02-24T18:35:33.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.980+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 >> 10.20.193.18:0/2597770642 conn(0x7f462c0ce730 msgr2=0x7f462c0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:33.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.980+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 shutdown_connections 2026-02-24T18:35:33.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.980+0000 7f4631e88640 1 -- 10.20.193.18:0/2597770642 wait complete. 2026-02-24T18:35:33.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.980+0000 7f4631e88640 1 librados: shutdown 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 10 monclient: get_monmap_and_config 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 10 monclient: build_initial_monmap 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 10 monclient: monmap: 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:33.991099+0000 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:33.991099+0000 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:33.991 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 1 Processor -- start 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 1 -- start start 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.990+0000 7f3f23353640 10 monclient: init 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient: _add_conns ranks=[0,3,5,4,2,1] 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3f1c1527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3f1c073530 0x7f3f1c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-a con 0x7f3f1c073530 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3f1c15f900 0x7f3f1c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-c con 0x7f3f1c15f900 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1c0739a0 con 0x7f3f1c1527f0 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1c074b10 con 0x7f3f1c073530 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1c15fd90 con 0x7f3f1c15f900 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.991+0000 7f3f23353640 10 monclient(hunting): authenticate will time out at 1477.672513s 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3f1c15f900 0x7f3f1c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3f1c073530 0x7f3f1c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:33.993 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:33.993 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3f1c15f900 0x7f3f1c0ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44198/0 (socket says 10.20.193.18:44198) 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 1 -- 10.20.193.18:0/2979210586 learned_addr learned my addr 10.20.193.18:0/2979210586 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): get_auth_request con 0x7f3f1c15f900 auth_method 0 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): get_auth_request con 0x7f3f1c073530 auth_method 0 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:33.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.992+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 10 monclient(hunting): handle_auth_done global_id 4451 payload 274 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 1 -- 10.20.193.18:0/2979210586 >> v2:10.20.193.18:3300/0 conn(0x7f3f1c073530 msgr2=0x7f3f1c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 1 --2- 10.20.193.18:0/2979210586 >> v2:10.20.193.18:3300/0 conn(0x7f3f1c073530 0x7f3f1c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 1 -- 10.20.193.18:0/2979210586 >> v1:10.20.193.18:6789/0 conn(0x7f3f1c1527f0 legacy=0x7f3f1c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 1 monclient: found mon.noname-c 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:33.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.993+0000 7f3f218c9640 1 -- 10.20.193.18:0/2979210586 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c15d9d0 con 0x7f3f1c15f900 2026-02-24T18:35:33.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.995+0000 7f3f218c9640 1 --2- 10.20.193.18:0/2979210586 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c0ccf70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c06cdf0 tx=0x7f3f1802f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e2df50b7dc0b00 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 1 -- 10.20.193.18:0/2979210586 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1802fcd0 con 0x7f3f1c15f900 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: dump: 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 1 -- 10.20.193.18:0/2979210586 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f180373f0 con 0x7f3f1c15f900 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 1 -- 10.20.193.18:0/2979210586 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180377c0 con 0x7f3f1c15f900 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f0bfff640 10 monclient: dump: 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f23353640 5 monclient: authenticate success, global_id 4451 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f23353640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:33.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.997+0000 7f3f23353640 4 set_mon_vals no callback set 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 10 monclient: shutdown 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 -- 10.20.193.18:0/2979210586 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 msgr2=0x7f3f1c0ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 --2- 10.20.193.18:0/2979210586 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c0ccf70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c06cdf0 tx=0x7f3f1802f270 comp rx=0 tx=0).stop 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 -- 10.20.193.18:0/2979210586 shutdown_connections 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 --2- 10.20.193.18:0/2979210586 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c0ccf70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 --2- 10.20.193.18:0/2979210586 >> v2:10.20.193.18:3300/0 conn(0x7f3f1c073530 0x7f3f1c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 -- 10.20.193.18:0/2979210586 >> 10.20.193.18:0/2979210586 conn(0x7f3f1c0ce730 msgr2=0x7f3f1c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 -- 10.20.193.18:0/2979210586 shutdown_connections 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.998+0000 7f3f23353640 1 -- 10.20.193.18:0/2979210586 wait complete. 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.999+0000 7f3f23353640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:33.999+0000 7f3f23353640 10 monclient: monmap: 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.000 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 librados: starting msgr at 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 librados: starting objecter 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 Processor -- start 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 -- start start 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 librados: setting wanted keys 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 1 librados: calling monclient init 2026-02-24T18:35:34.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.000+0000 7f3f23353640 10 monclient: init 2026-02-24T18:35:34.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:34.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 0x7f3f1c0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-b con 0x7f3f1c073530 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f1c1527f0 0x7f3f1c1f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-a con 0x7f3f1c1527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c1f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): picked mon.noname-c con 0x7f3f1c15f900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c0741e0 con 0x7f3f1c1527f0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c073f00 con 0x7f3f1c073530 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c0744c0 con 0x7f3f1c15f900 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f23353640 10 monclient(hunting): authenticate will time out at 1477.682728s 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f208c7640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f1c1527f0 0x7f3f1c1f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f210c8640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 0x7f3f1c0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.001+0000 7f3f218c9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c1f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 0x7f3f1c0c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53196/0 (socket says 10.20.193.18:53196) 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 1 -- 10.20.193.18:0/3676213211 learned_addr learned my addr 10.20.193.18:0/3676213211 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): get_auth_request con 0x7f3f1c073530 auth_method 0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): get_auth_request con 0x7f3f1c15f900 auth_method 0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f208c7640 10 monclient(hunting): get_auth_request con 0x7f3f1c1527f0 auth_method 0 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f208c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f208c7640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f208c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f218c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.002+0000 7f3f210c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f208c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f218c9640 10 monclient(hunting): handle_auth_done global_id 4454 payload 824 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f218c9640 1 -- 10.20.193.18:0/3676213211 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 msgr2=0x7f3f1c0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f218c9640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 0x7f3f1c0c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f218c9640 1 -- 10.20.193.18:0/3676213211 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f1c1527f0 msgr2=0x7f3f1c1f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.003+0000 7f3f218c9640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f1c1527f0 0x7f3f1c1f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.004+0000 7f3f218c9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.004+0000 7f3f218c9640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.004+0000 7f3f218c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.004+0000 7f3f218c9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.004+0000 7f3f218c9640 1 -- 10.20.193.18:0/3676213211 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c1f5db0 con 0x7f3f1c15f900 2026-02-24T18:35:34.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.005+0000 7f3f218c9640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c1f2070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f18035000 tx=0x7f3f18004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.005+0000 7f3f09ffb640 10 client.?.objecter ms_handle_connect 0x7f3f1c15f900 2026-02-24T18:35:34.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.005+0000 7f3f09ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18046020 con 0x7f3f1c15f900 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 10 monclient: dump: 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f18041660 con 0x7f3f1c15f900 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f23353640 5 monclient: authenticate success, global_id 4454 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f23353640 10 monclient: _renew_subs 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f23353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.006+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1f6090 con 0x7f3f1c15f900 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f09ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18009510 con 0x7f3f1c15f900 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f09ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f09ffb640 10 monclient: dump: 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.008 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f0b7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 client.4454.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 monclient: _renew_subs 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c1f1330 con 0x7f3f1c15f900 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 1 librados: init done 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.007+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c0cc490 con 0x7f3f1c15f900 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.008+0000 7fe6a5832640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.008+0000 7fe6a5832640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.008+0000 7fe6a5832640 10 monclient: monmap: 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.009841+0000 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.009841+0000 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.009+0000 7fe6a5832640 1 Processor -- start 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.009+0000 7fe6a5832640 1 -- start start 2026-02-24T18:35:34.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.009+0000 7fe6a5832640 10 monclient: init 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient: _add_conns ranks=[2,5,4,3,0,1] 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6a0155ef0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe6a01562d0 0x7fe6a015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-c con 0x7fe6a01562d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe6a00cde80 0x7fe6a00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-b con 0x7fe6a00cde80 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a00ce3b0 con 0x7fe6a0155ef0 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a015cdb0 con 0x7fe6a00cde80 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a00d2730 con 0x7fe6a01562d0 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe6a5832640 10 monclient(hunting): authenticate will time out at 1477.691261s 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe6a00cde80 0x7fe6a00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe6a00cde80 0x7fe6a00c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53200/0 (socket says 10.20.193.18:53200) 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 1 -- 10.20.193.18:0/675506358 learned_addr learned my addr 10.20.193.18:0/675506358 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 10 monclient(hunting): get_auth_request con 0x7fe6a00cde80 auth_method 0 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.010+0000 7fe69f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_done global_id 4453 payload 274 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 1 -- 10.20.193.18:0/675506358 >> v2:10.20.193.110:3300/0 conn(0x7fe6a01562d0 msgr2=0x7fe6a015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 1 --2- 10.20.193.18:0/675506358 >> v2:10.20.193.110:3300/0 conn(0x7fe6a01562d0 0x7fe6a015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 1 -- 10.20.193.18:0/675506358 >> v1:10.20.193.110:6789/0 conn(0x7fe6a0155ef0 legacy=0x7fe6a00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.011+0000 7fe69f7fe640 1 -- 10.20.193.18:0/675506358 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a00ce690 con 0x7fe6a00cde80 2026-02-24T18:35:34.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.013+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== osd.13 v2:10.20.193.32:6824/316721302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ffbbc024ec0 con 0x7ffba00813f0 2026-02-24T18:35:34.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.013+0000 7ffbb37fe640 10 client.4455.objecter ms_dispatch2 0x7ffbd4000b90 command_reply(tid 1: 0 ) 2026-02-24T18:35:34.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.013+0000 7ffbb37fe640 10 client.4455.objecter _finish_command 1 = system:0 2026-02-24T18:35:34.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.013+0000 7ffbb37fe640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 4 2026-02-24T18:35:34.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.013+0000 7ffbb37fe640 15 client.4455.objecter _session_command_op_remove 13 1 2026-02-24T18:35:34.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.014+0000 7fe69f7fe640 1 --2- 10.20.193.18:0/675506358 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a00cde80 0x7fe6a00c9080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6a008e920 tx=0x7fe69002f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c009b5227b08fb server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 1 -- 10.20.193.18:0/675506358 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002fd90 con 0x7fe6a00cde80 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: dump: 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 1 -- 10.20.193.18:0/675506358 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6900373f0 con 0x7fe6a00cde80 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 1 -- 10.20.193.18:0/675506358 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690037800 con 0x7fe6a00cde80 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe69dffb640 10 monclient: dump: 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.017 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe6a5832640 5 monclient: authenticate success, global_id 4453 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe6a5832640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.015+0000 7fe6a5832640 4 set_mon_vals no callback set 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 10 monclient: shutdown 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 -- 10.20.193.18:0/675506358 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a00cde80 msgr2=0x7fe6a00c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 --2- 10.20.193.18:0/675506358 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a00cde80 0x7fe6a00c9080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6a008e920 tx=0x7fe69002f330 comp rx=0 tx=0).stop 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 -- 10.20.193.18:0/675506358 shutdown_connections 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 --2- 10.20.193.18:0/675506358 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a00cde80 0x7fe6a00c9080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 --2- 10.20.193.18:0/675506358 >> v2:10.20.193.110:3300/0 conn(0x7fe6a01562d0 0x7fe6a015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 -- 10.20.193.18:0/675506358 >> 10.20.193.18:0/675506358 conn(0x7fe6a006d8c0 msgr2=0x7fe6a006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 -- 10.20.193.18:0/675506358 shutdown_connections 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.016+0000 7fe6a5832640 1 -- 10.20.193.18:0/675506358 wait complete. 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 10 monclient: monmap: 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 librados: starting msgr at 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 librados: starting objecter 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 Processor -- start 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 -- start start 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 librados: setting wanted keys 2026-02-24T18:35:34.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 1 librados: calling monclient init 2026-02-24T18:35:34.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.017+0000 7fe6a5832640 10 monclient: init 2026-02-24T18:35:34.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3f18041a30 con 0x7f3f1c15f900 2026-02-24T18:35:34.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6a00cde80 0x7fe6a01519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-c con 0x7fe6a00cde80 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-b con 0x7fe6a0155ef0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe6a01562d0 0x7fe6a014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): picked mon.noname-a con 0x7fe6a01562d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00cef30 con 0x7fe6a01562d0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00cec50 con 0x7fe6a0155ef0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00c95c0 con 0x7fe6a00cde80 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe6a5832640 10 monclient(hunting): authenticate will time out at 1477.699562s 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53208/0 (socket says 10.20.193.18:53208) 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 1 -- 10.20.193.18:0/1605784181 learned_addr learned my addr 10.20.193.18:0/1605784181 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 10 monclient(hunting): get_auth_request con 0x7fe6a0155ef0 auth_method 0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7fe69e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7f3f09ffb640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 0x7f3eec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7f3f210c8640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 0x7f3eec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.018+0000 7f3f210c8640 10 monclient: get_auth_request con 0x7f3eec078e50 auth_method 0 2026-02-24T18:35:34.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7f3f210c8640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 0x7f3eec07b310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c004660 tx=0x7f3f0c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7f3f09ffb640 10 client.4454.objecter ms_handle_connect 0x7f3eec078e50 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69f7fe640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe6a01562d0 0x7fe6a014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69f7fe640 10 monclient(hunting): get_auth_request con 0x7fe6a01562d0 auth_method 0 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.019+0000 7fe69f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f3f180027c0 con 0x7f3f1c15f900 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f3f09ffb640 10 client.4454.objecter ms_dispatch2 0x7f3f1c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f3f09ffb640 3 client.4454.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f3f09ffb640 3 client.4454.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f3f09ffb640 20 client.4454.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 10 monclient(hunting): handle_auth_done global_id 4456 payload 824 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 -- 10.20.193.18:0/1605784181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6a00cde80 msgr2=0x7fe6a01519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6a00cde80 0x7fe6a01519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 -- 10.20.193.18:0/1605784181 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe6a01562d0 msgr2=0x7fe6a014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe6a01562d0 0x7fe6a014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7fe69e7fc640 1 -- 10.20.193.18:0/1605784181 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01f3cb0 con 0x7fe6a0155ef0 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f2b27624640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f2b27624640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7f2b27624640 10 monclient: monmap: 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.021539+0000 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.021539+0000 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.023 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 10 client.4455.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 15 client.4455.objecter _session_command_op_assign -1 2 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 20 client.4455.objecter _get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 20 client.4455.objecter _get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 15 client.4455.objecter _session_command_op_remove -1 2 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 20 client.4455.objecter get_session s=0x7ffba0080d20 osd=13 3 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 15 client.4455.objecter _session_command_op_assign 13 2 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 4 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 10 client.4455.objecter _send_command 2 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.020+0000 7ffbd9334640 1 -- 10.20.193.18:0/2278326037 --> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffbd4150ef0 con 0x7ffba00813f0 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f2b27624640 1 Processor -- start 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f2b27624640 1 -- start start 2026-02-24T18:35:34.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f2b27624640 10 monclient: init 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f087ecf5640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f087ecf5640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.021+0000 7f087ecf5640 10 monclient: monmap: 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.022593+0000 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.022593+0000 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7fe69e7fc640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b500 tx=0x7fe68800b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7fe677fff640 10 client.?.objecter ms_handle_connect 0x7fe6a0155ef0 2026-02-24T18:35:34.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7fe677fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient: _add_conns ranks=[5,1,2,0,3,4] 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 Processor -- start 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f2b2015f650 0x7f2b2015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-c con 0x7f2b2015f650 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2b200ceca0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2b200c9b00 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b200d2450 con 0x7f2b200ceca0 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b200d2730 con 0x7f2b200c9b00 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2b200c8dd0 con 0x7f2b2015f650 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f2b27624640 10 monclient(hunting): authenticate will time out at 1477.703456s 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 -- start start 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient: init 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient: _add_conns ranks=[0,1,3,4,5,2] 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08781527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0878073530 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f087815fad0 0x7f08780ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-a con 0x7f087815fad0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08780739a0 con 0x7f08781527f0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f087815d9d0 con 0x7f0878073530 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0878074b10 con 0x7f087815fad0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.022+0000 7f087ecf5640 10 monclient(hunting): authenticate will time out at 1477.703965s 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004480 con 0x7fe6a0155ef0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: dump: 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7f1deca74640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7f1deca74640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7f1deca74640 10 monclient: monmap: 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.024550+0000 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.024550+0000 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.028 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe688014070 con 0x7fe6a0155ef0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe69d7fa640 4 set_mon_vals no callback set 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7f2b24b98640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f2b200ceca0 0x7f2b200d1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51198/0 (socket says 10.20.193.18:51198) 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7f2b24b98640 1 -- 10.20.193.18:0/3771736129 learned_addr learned my addr 10.20.193.18:0/3771736129 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe6a5832640 5 monclient: authenticate success, global_id 4456 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe6a5832640 10 monclient: _renew_subs 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe6a5832640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01f7fd0 con 0x7fe6a0155ef0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688002920 con 0x7fe6a0155ef0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.023+0000 7fe677fff640 10 monclient: dump: 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.029 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f086ffff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f0878073530 0x7f087815f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51206/0 (socket says 10.20.193.18:51206) 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 monclient: _renew_subs 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01f4230 con 0x7fe6a0155ef0 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 10 monclient: monmap: 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.024919+0000 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.024919+0000 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.030 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f086ffff640 1 -- 10.20.193.18:0/3430763671 learned_addr learned my addr 10.20.193.18:0/3430763671 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 1 librados: init done 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb37fe640 1 -- 10.20.193.18:0/2278326037 <== osd.13 v2:10.20.193.32:6824/316721302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffbd4150ef0 con 0x7ffba00813f0 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb37fe640 10 client.4455.objecter ms_dispatch2 0x7ffbd4000b90 command_reply(tid 2: 0 ) 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb37fe640 10 client.4455.objecter _finish_command 2 = system:0 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb37fe640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 4 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb37fe640 15 client.4455.objecter _session_command_op_remove 13 2 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f1deca74640 1 Processor -- start 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f087d26b640 1 --2- 10.20.193.18:0/3430763671 >> v2:10.20.193.18:3300/0 conn(0x7f087815fad0 0x7f08780ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f1deca74640 1 -- start start 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f1deca74640 10 monclient: init 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f087d26b640 10 monclient(hunting): get_auth_request con 0x7f087815fad0 auth_method 0 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f087d26b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f087d26b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f087d26b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 1 Processor -- start 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 10 librados: watch_flush enter 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 10 librados: watch_flush exit 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 10 client.4455.objecter close_session for osd.13 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7ffba00813f0 msgr2=0x7ffba0083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7ffba00813f0 0x7ffba0083830 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 20 client.4455.objecter put_session s=0x7ffba0080d20 osd=13 1 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 20 client.4455.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7ffbb17fa640 10 client.4455.objecter successfully canceled tick 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 1 -- start start 2026-02-24T18:35:34.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7f2e7f5d3640 10 monclient: init 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.024+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a00c9990 con 0x7fe6a0155ef0 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffba0078e50 msgr2=0x7ffba007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffba0078e50 0x7ffba007b310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005d00 tx=0x7ffbc4002e10 comp rx=0 tx=0).stop 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 10 monclient: shutdown 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 msgr2=0x7ffbd41f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 --2- 10.20.193.18:0/2278326037 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbd40cccb0 0x7ffbd41f43f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c490 tx=0x7ffbbc00c960 comp rx=0 tx=0).stop 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f087d26b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f087d26b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f087d26b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbd37fe640 1 -- 10.20.193.18:0/2278326037 reap_dead start 2026-02-24T18:35:34.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 shutdown_connections 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 >> 10.20.193.18:0/2278326037 conn(0x7ffbd40cc190 msgr2=0x7ffbd40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient: _add_conns ranks=[3,5,0,1,4,2] 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1de81527f0 0x7f1de8073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-a con 0x7f1de81527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1de815d2c0 0x7f1de815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient: _add_conns ranks=[1,4,3,2,5,0] 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 shutdown_connections 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-c con 0x7f1de815d2c0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1de815d730 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1de8073c20 con 0x7f1de815d730 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1de8074b10 con 0x7f1de81527f0 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1de8073f00 con 0x7f1de815d2c0 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f1deca74640 10 monclient(hunting): authenticate will time out at 1477.706392s 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2e781527f0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2e78073530 0x7f2e7815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-b con 0x7f2e78073530 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2e7815f900 0x7f2e780ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-a con 0x7f2e7815f900 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e780739a0 con 0x7f2e781527f0 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2e78074b10 con 0x7f2e7815f900 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e7815fd90 con 0x7f2e78073530 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7f2e7f5d3640 10 monclient(hunting): authenticate will time out at 1477.706393s 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.025+0000 7ffbb17fa640 1 -- 10.20.193.18:0/2278326037 wait complete. 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7ffbb17fa640 1 librados: shutdown 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 10 monclient(hunting): handle_auth_done global_id 4476 payload 274 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 1 -- 10.20.193.18:0/3430763671 >> v1:10.20.193.32:6789/0 conn(0x7f0878073530 legacy=0x7f087815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 1 -- 10.20.193.18:0/3430763671 >> v1:10.20.193.18:6789/0 conn(0x7f08781527f0 legacy=0x7f087815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.026+0000 7f087d26b640 1 -- 10.20.193.18:0/3430763671 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0878073c20 con 0x7f087815fad0 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.028+0000 7f821a9d3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.028+0000 7f821a9d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.028+0000 7f821a9d3640 10 monclient: monmap: 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.029694+0000 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.029694+0000 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f087d26b640 1 --2- 10.20.193.18:0/3430763671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f087815fad0 0x7f08780ccf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f7661f32c764db server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 1 Processor -- start 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 1 -- start start 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 10 monclient: init 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 10 monclient: _add_conns ranks=[4,3,2,0,1,5] 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f82141527f0 0x7f8214073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-b con 0x7f82141527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f821415d2c0 0x7f821415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-a con 0x7f821415d2c0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.029+0000 7f1de6575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1de81527f0 0x7f1de8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f821415d730 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8214073c20 con 0x7f821415d730 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8214074b10 con 0x7f821415d2c0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8214073f00 con 0x7f82141527f0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f821a9d3640 10 monclient(hunting): authenticate will time out at 1477.711025s 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1de81527f0 0x7f1de8073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41418/0 (socket says 10.20.193.18:41418) 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 1 -- 10.20.193.18:0/2926155163 learned_addr learned my addr 10.20.193.18:0/2926155163 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 10 monclient(hunting): get_auth_request con 0x7f1de81527f0 auth_method 0 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.030+0000 7f1de6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 10 monclient: monmap: 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.032279+0000 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.032279+0000 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 1 Processor -- start 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 1 -- start start 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.031+0000 7efc0b1dc640 10 monclient: init 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 10 monclient(hunting): handle_auth_done global_id 4482 payload 274 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 1 -- 10.20.193.18:0/2926155163 >> v2:10.20.193.110:3300/0 conn(0x7f1de815d2c0 msgr2=0x7f1de815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 1 --2- 10.20.193.18:0/2926155163 >> v2:10.20.193.110:3300/0 conn(0x7f1de815d2c0 0x7f1de815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 1 -- 10.20.193.18:0/2926155163 >> v1:10.20.193.18:6789/0 conn(0x7f1de815d730 legacy=0x7f1de80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7f1de6575640 1 -- 10.20.193.18:0/2926155163 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de80741e0 con 0x7f1de81527f0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient: _add_conns ranks=[0,4,2,1,5,3] 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efc041527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efc04073530 0x7efc0415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-b con 0x7efc04073530 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efc0415f900 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc040739a0 con 0x7efc041527f0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0415fd90 con 0x7efc0415f900 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7efc04074b10 con 0x7efc04073530 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.032+0000 7efc0b1dc640 10 monclient(hunting): authenticate will time out at 1477.713890s 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2e7815f900 0x7f2e780ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2e7815f900 0x7f2e780ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41424/0 (socket says 10.20.193.18:41424) 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 1 -- 10.20.193.18:0/3988535906 learned_addr learned my addr 10.20.193.18:0/3988535906 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 10 monclient(hunting): get_auth_request con 0x7f2e7815f900 auth_method 0 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.033+0000 7f2e7db49640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f1de6575640 1 --2- 10.20.193.18:0/2926155163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de8073530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1de808f510 tx=0x7f1ddc02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf965db1536d03d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f1f5ae58640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f1f5ae58640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f1f5ae58640 10 monclient: monmap: 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.035569+0000 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.035569+0000 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f451d911640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f451d911640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.034+0000 7f451d911640 10 monclient: monmap: 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.035580+0000 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.035580+0000 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.040 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 Processor -- start 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 1 Processor -- start 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 -- start start 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient: init 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 1 -- start start 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 10 monclient: init 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 10 monclient(hunting): handle_auth_done global_id 4485 payload 274 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 1 -- 10.20.193.18:0/3988535906 >> v2:10.20.193.32:3300/0 conn(0x7f2e78073530 msgr2=0x7f2e7815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 1 --2- 10.20.193.18:0/3988535906 >> v2:10.20.193.32:3300/0 conn(0x7f2e78073530 0x7f2e7815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 1 -- 10.20.193.18:0/3988535906 >> v1:10.20.193.32:6789/0 conn(0x7f2e781527f0 legacy=0x7f2e7815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f2e7db49640 1 -- 10.20.193.18:0/3988535906 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7815d9d0 con 0x7f2e7815f900 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient: _add_conns ranks=[5,1,2,4,0,3] 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4518152b30 0x7f451815b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): picked mon.noname-c con 0x7f4518152b30 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4518152f10 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f451815c730 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4518159fc0 con 0x7f4518152f10 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451815a2a0 con 0x7f451815c730 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f451815a580 con 0x7f4518152b30 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f451d911640 10 monclient(hunting): authenticate will time out at 1477.716747s 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7fc896198640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 10 monclient: _add_conns ranks=[0,5,2,4,1,3] 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7fc896198640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7fc896198640 10 monclient: monmap: 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.036616+0000 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.036616+0000 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.042 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1f541527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.035+0000 7f1f5ae58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1f54073530 0x7f1f5415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-c con 0x7f1f54073530 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1f5415f900 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f540739a0 con 0x7f1f541527f0 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5415fd90 con 0x7f1f5415f900 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1f54074b10 con 0x7f1f54073530 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 Processor -- start 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 -- start start 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient: init 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7efc08f51640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7efc041527f0 0x7efc0415d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58580/0 (socket says 10.20.193.18:58580) 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7efc08f51640 1 -- 10.20.193.18:0/2517153324 learned_addr learned my addr 10.20.193.18:0/2517153324 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f1f5ae58640 10 monclient(hunting): authenticate will time out at 1477.717372s 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient: _add_conns ranks=[4,2,3,1,5,0] 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc8901590a0 0x7fc890158730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): picked mon.noname-b con 0x7fc8901590a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7f82137fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f821415d2c0 0x7f821415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc890072480 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8900739e0 0x7fc8900c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): picked mon.noname-a con 0x7fc8900739e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc890072980 con 0x7fc890072480 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc89015afb0 con 0x7fc8900739e0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc890073550 con 0x7fc8901590a0 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.036+0000 7fc896198640 10 monclient(hunting): authenticate will time out at 1477.717820s 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:34.044 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f821415d2c0 0x7f821415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41430/0 (socket says 10.20.193.18:41430) 2026-02-24T18:35:34.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 1 -- 10.20.193.18:0/377819594 learned_addr learned my addr 10.20.193.18:0/377819594 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 10 monclient(hunting): get_auth_request con 0x7f821415d2c0 auth_method 0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7f82137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.037+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe688019460 con 0x7fe6a0155ef0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7fe677fff640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 0x7fe67007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7fe69effd640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 0x7fe67007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7fe69effd640 10 monclient: get_auth_request con 0x7fe670078e50 auth_method 0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7fe69effd640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 0x7fe67007b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.038+0000 7fe677fff640 10 client.4456.objecter ms_handle_connect 0x7fe670078e50 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fe68805ddb0 con 0x7fe6a0155ef0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7fe677fff640 10 client.4456.objecter ms_dispatch2 0x7fe6a0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7fe677fff640 3 client.4456.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7fe677fff640 3 client.4456.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7fe677fff640 20 client.4456.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 10 monclient(hunting): handle_auth_done global_id 4488 payload 274 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 1 -- 10.20.193.18:0/377819594 >> v2:10.20.193.32:3300/0 conn(0x7f82141527f0 msgr2=0x7f8214073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 1 --2- 10.20.193.18:0/377819594 >> v2:10.20.193.32:3300/0 conn(0x7f82141527f0 0x7f8214073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 1 -- 10.20.193.18:0/377819594 >> v1:10.20.193.110:6789/0 conn(0x7f821415d730 legacy=0x7f82140ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.040+0000 7f82137fe640 1 -- 10.20.193.18:0/377819594 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82140741e0 con 0x7f821415d2c0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 1 -- 10.20.193.18:0/3430763671 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087403c070 con 0x7f087815fad0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: dump: 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f9b64bb3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 1 -- 10.20.193.18:0/3430763671 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0874037440 con 0x7f087815fad0 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 1 -- 10.20.193.18:0/3430763671 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874037810 con 0x7f087815fad0 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f086f7fe640 10 monclient: dump: 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f087ecf5640 5 monclient: authenticate success, global_id 4476 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f087ecf5640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f087ecf5640 4 set_mon_vals no callback set 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f9b64bb3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.041+0000 7f9b64bb3640 10 monclient: monmap: 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.042676+0000 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.042676+0000 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 Processor -- start 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 10 monclient: shutdown 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 -- start start 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient: init 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 -- 10.20.193.18:0/3430763671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f087815fad0 msgr2=0x7f08780ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3430763671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f087815fad0 0x7f08780ccf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402f230 comp rx=0 tx=0).stop 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 -- 10.20.193.18:0/3430763671 shutdown_connections 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3430763671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f087815fad0 0x7f08780ccf70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 -- 10.20.193.18:0/3430763671 >> 10.20.193.18:0/3430763671 conn(0x7f08780ce730 msgr2=0x7f08780ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 -- 10.20.193.18:0/3430763671 shutdown_connections 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f087ecf5640 1 -- 10.20.193.18:0/3430763671 wait complete. 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient: _add_conns ranks=[3,4,0,5,2,1] 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 0x7f9b60073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-a con 0x7f9b601527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9b6015d2c0 0x7f9b6015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-b con 0x7f9b6015d2c0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b6015d730 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b60073c20 con 0x7f9b6015d730 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9b60074b10 con 0x7f9b601527f0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9b60073f00 con 0x7f9b6015d2c0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.042+0000 7f9b64bb3640 10 monclient(hunting): authenticate will time out at 1477.723939s 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient: monmap: 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 librados: starting msgr at 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 librados: starting objecter 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 Processor -- start 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f2e7db49640 1 --2- 10.20.193.18:0/3988535906 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e7815f900 0x7f2e780ccf70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e7808f510 tx=0x7f2e7402f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add3d519254a23b8 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 -- start start 2026-02-24T18:35:34.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 librados: setting wanted keys 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 librados: calling monclient init 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient: init 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f82137fe640 1 --2- 10.20.193.18:0/377819594 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f821415d2c0 0x7f821415f3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f821406e9a0 tx=0x7f820402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcb8d724638dad0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-a con 0x7f0878073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08781527f0 0x7f08780c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-b con 0x7f08781527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f087815fad0 0x7f08781f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f5a3f4f5640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f5a3f4f5640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f5a3f4f5640 10 monclient: monmap: 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.044836+0000 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.044836+0000 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.050 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7f1f58bcd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1f541527f0 0x7f1f5415d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58596/0 (socket says 10.20.193.18:58596) 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7f1f58bcd640 1 -- 10.20.193.18:0/3369297776 learned_addr learned my addr 10.20.193.18:0/3369297776 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): picked mon.noname-c con 0x7f087815fad0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314085382 0 0) 0x7efbf4002920 con 0x7efc041527f0 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 10 monclient(hunting): my global_id is 4491 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbf0003620 con 0x7efc041527f0 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f08780744c0 con 0x7f0878073530 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08780741e0 con 0x7f08781527f0 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08780747a0 con 0x7f087815fad0 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.043+0000 7f087ecf5640 10 monclient(hunting): authenticate will time out at 1477.725006s 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7f5a3f4f5640 1 Processor -- start 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7f5a3f4f5640 1 -- start start 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.044+0000 7f5a3f4f5640 10 monclient: init 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient: _add_conns ranks=[0,1,3,5,2,4] 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5a381527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5a38073530 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f5a3815fad0 0x7f5a380ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-a con 0x7f5a3815fad0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a380739a0 con 0x7f5a381527f0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3815d9d0 con 0x7f5a38073530 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5a38074b10 con 0x7f5a3815fad0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f5a3f4f5640 10 monclient(hunting): authenticate will time out at 1477.726282s 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 1 -- 10.20.193.18:0/2926155163 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc03c070 con 0x7f1de81527f0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: dump: 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.052 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 1 -- 10.20.193.18:0/2926155163 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc02fb40 con 0x7f1de81527f0 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 1 -- 10.20.193.18:0/2926155163 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc037440 con 0x7f1de81527f0 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1de5573640 10 monclient: dump: 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 5 monclient: authenticate success, global_id 4482 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 4 set_mon_vals no callback set 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7fc88ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8900739e0 0x7fc8900c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 10 monclient: shutdown 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 -- 10.20.193.18:0/2926155163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 msgr2=0x7f1de8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 --2- 10.20.193.18:0/2926155163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de8073530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1de808f510 tx=0x7f1ddc02eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 -- 10.20.193.18:0/2926155163 shutdown_connections 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 --2- 10.20.193.18:0/2926155163 >> v2:10.20.193.110:3300/0 conn(0x7f1de815d2c0 0x7f1de815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 --2- 10.20.193.18:0/2926155163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de8073530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.045+0000 7f1deca74640 1 -- 10.20.193.18:0/2926155163 >> 10.20.193.18:0/2926155163 conn(0x7f1de80ce730 msgr2=0x7f1de80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7f1deca74640 1 -- 10.20.193.18:0/2926155163 shutdown_connections 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7f1deca74640 1 -- 10.20.193.18:0/2926155163 wait complete. 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8900739e0 0x7fc8900c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41436/0 (socket says 10.20.193.18:41436) 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 1 -- 10.20.193.18:0/2184163523 learned_addr learned my addr 10.20.193.18:0/2184163523 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 10 monclient(hunting): get_auth_request con 0x7fc8900739e0 auth_method 0 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7fc88ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7f1deca74640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.046+0000 7f1deca74640 10 monclient: monmap: 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.054 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 librados: starting msgr at 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 librados: starting objecter 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 Processor -- start 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 -- start start 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 librados: setting wanted keys 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 librados: calling monclient init 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient: init 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-a con 0x7f1de81527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1de815d2c0 0x7f1de80c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-b con 0x7f1de815d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1de815d730 0x7f1de81f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): picked mon.noname-c con 0x7f1de815d730 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8074fa0 con 0x7f1de81527f0 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1de80747a0 con 0x7f1de815d2c0 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8075120 con 0x7f1de815d730 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.047+0000 7f1deca74640 10 monclient(hunting): authenticate will time out at 1477.728709s 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 1 -- 10.20.193.18:0/3988535906 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402fcd0 con 0x7f2e7815f900 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: dump: 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 1 -- 10.20.193.18:0/3988535906 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e740373f0 con 0x7f2e7815f900 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e7f5d3640 5 monclient: authenticate success, global_id 4485 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e7f5d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 1 -- 10.20.193.18:0/3988535906 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e740377c0 con 0x7f2e7815f900 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.051+0000 7f2e67fff640 10 monclient: dump: 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.056 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 4 set_mon_vals no callback set 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2078389144 0 0) 0x7efbf0003620 con 0x7efc041527f0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 >> v2:10.20.193.32:3300/0 conn(0x7efc04073530 msgr2=0x7efc0415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 --2- 10.20.193.18:0/2517153324 >> v2:10.20.193.32:3300/0 conn(0x7efc04073530 0x7efc0415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 >> v1:10.20.193.110:6789/0 conn(0x7efc0415f900 legacy=0x7efc040ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0415d9d0 con 0x7efc041527f0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7efbf40030d0 con 0x7efc041527f0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: dump: 2026-02-24T18:35:34.057 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 >> v1:10.20.193.18:6789/0 conn(0x7efc041527f0 legacy=0x7efc0415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 10 monclient: shutdown 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efbf0003ba0 0x7efbf0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): picked mon.b con 0x7efbf0003ba0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 0x7efbf0004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): picked mon.a con 0x7efbf0004900 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0005510 0x7efbf000bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/3988535906 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e7815f900 msgr2=0x7f2e780ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): picked mon.c con 0x7efbf0005510 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efc04074b10 con 0x7efbf0004900 2026-02-24T18:35:34.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efbf000c450 con 0x7efbf0003ba0 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efbf000c650 con 0x7efbf0005510 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7efbfb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/3988535906 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e7815f900 0x7f2e780ccf70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e7808f510 tx=0x7f2e7402f270 comp rx=0 tx=0).stop 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/3988535906 shutdown_connections 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/3988535906 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e7815f900 0x7f2e780ccf70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/3988535906 >> v2:10.20.193.32:3300/0 conn(0x7f2e78073530 0x7f2e7815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/3988535906 >> 10.20.193.18:0/3988535906 conn(0x7f2e780ce730 msgr2=0x7f2e780ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/3988535906 shutdown_connections 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.052+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/3988535906 wait complete. 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7f2e7f5d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 10 monclient(hunting): handle_auth_done global_id 4497 payload 274 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 1 -- 10.20.193.18:0/2184163523 >> v2:10.20.193.32:3300/0 conn(0x7fc8901590a0 msgr2=0x7fc890158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 1 --2- 10.20.193.18:0/2184163523 >> v2:10.20.193.32:3300/0 conn(0x7fc8901590a0 0x7fc890158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 1 -- 10.20.193.18:0/2184163523 >> v1:10.20.193.110:6789/0 conn(0x7fc890072480 legacy=0x7fc890151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7f2e7f5d3640 10 monclient: monmap: 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.059 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.053+0000 7fc88ffff640 1 -- 10.20.193.18:0/2184163523 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc890072c20 con 0x7fc8900739e0 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 librados: starting msgr at 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 librados: starting objecter 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 Processor -- start 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 -- start start 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 librados: setting wanted keys 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 librados: calling monclient init 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient: init 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2e78073530 0x7f2e781f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-c con 0x7f2e78073530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e781527f0 0x7f2e781f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-a con 0x7f2e781527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): picked mon.noname-b con 0x7f2e7815f900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780741e0 con 0x7f2e781527f0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78073f00 con 0x7f2e7815f900 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780744c0 con 0x7f2e78073530 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.054+0000 7f2e7f5d3640 10 monclient(hunting): authenticate will time out at 1477.735850s 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7fc88ffff640 1 --2- 10.20.193.18:0/2184163523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8900c7d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8900736d0 tx=0x7fc88402f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec4b386ecbd5e9b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 1 -- 10.20.193.18:0/3369297776 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239598762 0 0) 0x7f1f44002920 con 0x7f1f541527f0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 10 monclient(hunting): my global_id is 4494 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.055+0000 7f1f4b7fe640 1 -- 10.20.193.18:0/3369297776 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f40003620 con 0x7f1f541527f0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.056+0000 7f1de6575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.057+0000 7f5bede0b640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.057+0000 7f5bede0b640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.057+0000 7f5bede0b640 10 monclient: monmap: 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.058526+0000 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.058526+0000 2026-02-24T18:35:34.061 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 Processor -- start 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 -- start start 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient: init 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient: _add_conns ranks=[2,3,5,4,0,1] 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5be81527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f5be8073530 0x7f5be815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-a con 0x7f5be8073530 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5be815f900 0x7f5be80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 0x7f9b60073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-c con 0x7f5be815f900 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be80739a0 con 0x7f5be81527f0 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5be8074b10 con 0x7f5be8073530 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5be815fd90 con 0x7f5be815f900 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f5bede0b640 10 monclient(hunting): authenticate will time out at 1477.739862s 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 0x7f9b60073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41448/0 (socket says 10.20.193.18:41448) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 1 -- 10.20.193.18:0/2874150314 learned_addr learned my addr 10.20.193.18:0/2874150314 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 10 monclient(hunting): get_auth_request con 0x7f9b601527f0 auth_method 0 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.058+0000 7f9b5e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f087ca6a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41474/0 (socket says 10.20.193.18:41474) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 1 -- 10.20.193.18:0/2932026534 learned_addr learned my addr 10.20.193.18:0/2932026534 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 10 monclient(hunting): get_auth_request con 0x7f1de81527f0 auth_method 0 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.059+0000 7f1de6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 1 -- 10.20.193.18:0/377819594 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820403c070 con 0x7f821415d2c0 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: dump: 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.063 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 5 monclient: authenticate success, global_id 4488 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 1 -- 10.20.193.18:0/377819594 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820402fb40 con 0x7f821415d2c0 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f1de6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7efbfbfff640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 0x7efbf0004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 1 -- 10.20.193.18:0/377819594 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820403c070 con 0x7f821415d2c0 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f8212ffd640 10 monclient: dump: 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 4 set_mon_vals no callback set 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 10 monclient: shutdown 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 -- 10.20.193.18:0/377819594 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f821415d2c0 msgr2=0x7f821415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 --2- 10.20.193.18:0/377819594 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f821415d2c0 0x7f821415f3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f821406e9a0 tx=0x7f820402eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 -- 10.20.193.18:0/377819594 shutdown_connections 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 --2- 10.20.193.18:0/377819594 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f821415d2c0 0x7f821415f3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 --2- 10.20.193.18:0/377819594 >> v2:10.20.193.32:3300/0 conn(0x7f82141527f0 0x7f8214073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 -- 10.20.193.18:0/377819594 >> 10.20.193.18:0/377819594 conn(0x7f82140ce730 msgr2=0x7f82140ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 -- 10.20.193.18:0/377819594 shutdown_connections 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.060+0000 7f821a9d3640 1 -- 10.20.193.18:0/377819594 wait complete. 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 10 monclient: monmap: 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 librados: starting msgr at 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 librados: starting objecter 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 Processor -- start 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 -- start start 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 librados: setting wanted keys 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 1 librados: calling monclient init 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.061+0000 7f821a9d3640 10 monclient: init 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82141527f0 0x7f821414e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-a con 0x7f82141527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-c con 0x7f821415d2c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f821415d730 0x7f82140c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): picked mon.noname-b con 0x7f821415d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8214074fa0 con 0x7f82141527f0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82140747a0 con 0x7f821415d730 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8214075120 con 0x7f821415d2c0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f821a9d3640 10 monclient(hunting): authenticate will time out at 1477.743151s 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 10 monclient(hunting): handle_auth_done global_id 4503 payload 824 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 -- 10.20.193.18:0/2932026534 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1de815d730 msgr2=0x7f1de81f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1de815d730 0x7f1de81f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 -- 10.20.193.18:0/2932026534 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1de815d2c0 msgr2=0x7f1de80c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1de815d2c0 0x7f1de80c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.062+0000 7f1de6575640 1 -- 10.20.193.18:0/2932026534 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de80ca570 con 0x7f1de81527f0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f1de6575640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009a70 tx=0x7f1ddc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f1dceffd640 10 client.?.objecter ms_handle_connect 0x7f1de81527f0 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f1dceffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41462/0 (socket says 10.20.193.18:41462) 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 1 -- 10.20.193.18:0/3054785757 learned_addr learned my addr 10.20.193.18:0/3054785757 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 10 monclient(hunting): get_auth_request con 0x7f0878073530 auth_method 0 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.063+0000 7f087ca6a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7efbfbfff640 10 monclient(hunting): get_auth_request con 0x7efbf0004900 auth_method 0 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7efbfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7efbfbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7efbfbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7f087ca6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7f087ca6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.064+0000 7f087ca6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.065+0000 7efbfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.065+0000 7efbfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.065+0000 7efbfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.065+0000 7f5a3d26a640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f5a381527f0 0x7f5a3815d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58614/0 (socket says 10.20.193.18:58614) 2026-02-24T18:35:34.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.065+0000 7f5a3d26a640 1 -- 10.20.193.18:0/480914891 learned_addr learned my addr 10.20.193.18:0/480914891 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 10 monclient(hunting): handle_auth_done global_id 4506 payload 824 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 -- 10.20.193.18:0/3054785757 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f087815fad0 msgr2=0x7f08781f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f087815fad0 0x7f08781f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 -- 10.20.193.18:0/3054785757 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08781527f0 msgr2=0x7f08780c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08781527f0 0x7f08780c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.068+0000 7f087ca6a640 1 -- 10.20.193.18:0/3054785757 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781f41b0 con 0x7f0878073530 2026-02-24T18:35:34.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 10 monclient(hunting): handle_auth_done global_id 4491 payload 293 2026-02-24T18:35:34.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 -- 10.20.193.18:0/2517153324 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0005510 msgr2=0x7efbf000bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0005510 0x7efbf000bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 -- 10.20.193.18:0/2517153324 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efbf0003ba0 msgr2=0x7efbf0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efbf0003ba0 0x7efbf0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 monclient: found mon.a 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efc09752640 1 -- 10.20.193.18:0/2517153324 reap_dead start 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 1 -- 10.20.193.18:0/2517153324 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf0003620 con 0x7efbf0004900 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efbfbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efc0b1dc640 5 monclient: authenticate success, global_id 4491 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.069+0000 7efc0b1dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.070+0000 7f184c861640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.070+0000 7f184c861640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.070+0000 7f184c861640 10 monclient: monmap: 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.071443+0000 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.071443+0000 2026-02-24T18:35:34.071 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 Processor -- start 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 -- start start 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient: init 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f087ca6a640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f086800c700 tx=0x7f086800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f086d7fa640 10 client.?.objecter ms_handle_connect 0x7f0878073530 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f086d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient: _add_conns ranks=[0,5,1,2,4,3] 2026-02-24T18:35:34.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18441527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1844152bb0 0x7f18440c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): picked mon.noname-c con 0x7f1844152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f18440c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184406e300 con 0x7f18441527d0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184406e5e0 con 0x7f18440c5fc0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f184406f1f0 con 0x7f1844152bb0 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.071+0000 7f184c861640 10 monclient(hunting): authenticate will time out at 1477.752696s 2026-02-24T18:35:34.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.072+0000 7efbfbfff640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 0x7efbf0004d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efbec009980 tx=0x7efbec02ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4209eab62df3d2 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.073+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/480914891 >> v2:10.20.193.18:3300/0 conn(0x7f5a3815fad0 0x7f5a380ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 1 -- 10.20.193.18:0/2184163523 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88402fce0 con 0x7fc8900739e0 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: dump: 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 1 -- 10.20.193.18:0/2184163523 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8840373f0 con 0x7fc8900739e0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 1 -- 10.20.193.18:0/2184163523 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8840377c0 con 0x7fc8900739e0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 5 monclient: authenticate success, global_id 4497 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 4 set_mon_vals no callback set 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc88e7fc640 10 monclient: dump: 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a3da6b640 10 monclient(hunting): get_auth_request con 0x7f5a3815fad0 auth_method 0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a3da6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a3da6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a3da6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc046070 con 0x7f1de81527f0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: dump: 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc041890 con 0x7f1de81527f0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 5 monclient: authenticate success, global_id 4503 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 10 monclient: _renew_subs 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de80c91f0 con 0x7f1de81527f0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc03c040 con 0x7f1de81527f0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1dceffd640 10 monclient: dump: 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1de4d72640 4 set_mon_vals no callback set 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 10 client.4503.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 10 monclient: _renew_subs 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de80c97b0 con 0x7f1de81527f0 2026-02-24T18:35:34.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f1deca74640 1 librados: init done 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 10 monclient: shutdown 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 -- 10.20.193.18:0/2184163523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 msgr2=0x7fc8900c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 --2- 10.20.193.18:0/2184163523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8900c7d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8900736d0 tx=0x7fc88402f280 comp rx=0 tx=0).stop 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 -- 10.20.193.18:0/2184163523 shutdown_connections 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 --2- 10.20.193.18:0/2184163523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8900c7d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 --2- 10.20.193.18:0/2184163523 >> v2:10.20.193.32:3300/0 conn(0x7fc8901590a0 0x7fc890158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 -- 10.20.193.18:0/2184163523 >> 10.20.193.18:0/2184163523 conn(0x7fc8900cd780 msgr2=0x7fc8900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 -- 10.20.193.18:0/2184163523 shutdown_connections 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7fc896198640 1 -- 10.20.193.18:0/2184163523 wait complete. 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 1 -- 10.20.193.18:0/480914891 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233027710 0 0) 0x7f5a28002920 con 0x7f5a381527f0 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 10 monclient(hunting): _negotiate 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 10 monclient(hunting): my global_id is 4509 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.076+0000 7f5a27fff640 1 -- 10.20.193.18:0/480914891 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a20003620 con 0x7f5a381527f0 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f1deca74640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f1deca74640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f1deca74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 10 monclient: monmap: 2026-02-24T18:35:34.080 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de8152bd0 con 0x7f1de81527f0 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 librados: starting msgr at 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 librados: starting objecter 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 Processor -- start 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 -- start start 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 librados: setting wanted keys 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 1 librados: calling monclient init 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.077+0000 7fc896198640 10 monclient: init 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): picked mon.noname-b con 0x7fc890072480 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8901f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): picked mon.noname-a con 0x7fc8900739e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8901590a0 0x7fc8901f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): picked mon.noname-c con 0x7fc8901590a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8900c8c80 con 0x7fc8900739e0 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8900731e0 con 0x7fc890072480 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc890073e50 con 0x7fc8901590a0 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.078+0000 7fc896198640 10 monclient(hunting): authenticate will time out at 1477.759056s 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.118+0000 7f90eff96640 10 monclient: get_monmap_and_config 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.118+0000 7f90eff96640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.118+0000 7f90eff96640 10 monclient: monmap: 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:34.119835+0000 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:34.119835+0000 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 Processor -- start 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 -- start start 2026-02-24T18:35:34.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient: init 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient: _add_conns ranks=[4,2,5,1,0,3] 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f90e81527d0 0x7f90e806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-b con 0x7f90e81527d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f90e8152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f90e80c6190 0x7f90e80c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-c con 0x7f90e80c6190 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e806e300 con 0x7f90e8152bb0 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f90e806f1f0 con 0x7f90e81527d0 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f90e806e5e0 con 0x7f90e80c6190 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.119+0000 7f90eff96640 10 monclient(hunting): authenticate will time out at 1477.800957s 2026-02-24T18:35:34.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.121+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3f18050030 con 0x7f3f1c15f900 2026-02-24T18:35:34.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.121+0000 7f3f09ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.121+0000 7f3f09ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4518152b30 0x7f451815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4518152b30 0x7f451815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44236/0 (socket says 10.20.193.18:44236) 2026-02-24T18:35:34.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 1 -- 10.20.193.18:0/2143703699 learned_addr learned my addr 10.20.193.18:0/2143703699 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): get_auth_request con 0x7f4518152b30 auth_method 0 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.124+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.125+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2026-02-24T18:35:34.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 1 -- 10.20.193.18:0/2143703699 >> v1:10.20.193.110:6789/0 conn(0x7f451815c730 legacy=0x7f45180c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 1 -- 10.20.193.18:0/2143703699 >> v1:10.20.193.32:6789/0 conn(0x7f4518152f10 legacy=0x7f451815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.126+0000 7f4516ffd640 1 -- 10.20.193.18:0/2143703699 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451815a860 con 0x7f4518152b30 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5be815f900 0x7f5be80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f5be815f900 0x7f5be80ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44314/0 (socket says 10.20.193.18:44314) 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 1 -- 10.20.193.18:0/2582020374 learned_addr learned my addr 10.20.193.18:0/2582020374 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 10 monclient(hunting): get_auth_request con 0x7f5be815f900 auth_method 0 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f5be7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.127+0000 7f4516ffd640 1 --2- 10.20.193.18:0/2143703699 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4518152b30 0x7f451815b340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f450c0099a0 tx=0x7f450c02efe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b57e8b7c3bb38f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.128+0000 7f5be7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.128+0000 7f5be7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.128+0000 7f5be7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 10 monclient(hunting): handle_auth_done global_id 4460 payload 274 2026-02-24T18:35:34.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 1 -- 10.20.193.18:0/2582020374 >> v2:10.20.193.18:3300/0 conn(0x7f5be8073530 msgr2=0x7f5be815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 1 --2- 10.20.193.18:0/2582020374 >> v2:10.20.193.18:3300/0 conn(0x7f5be8073530 0x7f5be815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 1 -- 10.20.193.18:0/2582020374 >> v1:10.20.193.110:6789/0 conn(0x7f5be81527f0 legacy=0x7f5be815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.129+0000 7f5be7fff640 1 -- 10.20.193.18:0/2582020374 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be815d9d0 con 0x7f5be815f900 2026-02-24T18:35:34.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.130+0000 7f1f4bfff640 1 --2- 10.20.193.18:0/3369297776 >> v2:10.20.193.110:3300/0 conn(0x7f1f54073530 0x7f1f5415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f2b25399640 1 --2- 10.20.193.18:0/3771736129 >> v2:10.20.193.110:3300/0 conn(0x7f2b2015f650 0x7f2b2015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f1f4bfff640 10 monclient(hunting): get_auth_request con 0x7f1f54073530 auth_method 0 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f1f4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f1f4bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f1f4bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f2b25399640 10 monclient(hunting): get_auth_request con 0x7f2b2015f650 auth_method 0 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f2b25399640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f2b25399640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.131+0000 7f2b25399640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.132+0000 7f5be7fff640 1 --2- 10.20.193.18:0/2582020374 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be80ccf70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5be806cdf0 tx=0x7f5bdc02f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2cc0398b4e6244 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.132+0000 7f1f4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.132+0000 7f1f4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.132+0000 7f1f4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.133+0000 7f2b25399640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.133+0000 7f2b25399640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.133+0000 7f2b25399640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.133+0000 7f82137fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44318/0 (socket says 10.20.193.18:44318) 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 1 -- 10.20.193.18:0/1025406548 learned_addr learned my addr 10.20.193.18:0/1025406548 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 10 monclient(hunting): get_auth_request con 0x7f821415d2c0 auth_method 0 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.134+0000 7f82137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 10 monclient(hunting): handle_auth_done global_id 4463 payload 274 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 1 -- 10.20.193.18:0/3369297776 >> v1:10.20.193.110:6789/0 conn(0x7f1f5415f900 legacy=0x7f1f540ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 1 -- 10.20.193.18:0/3369297776 >> v1:10.20.193.18:6789/0 conn(0x7f1f541527f0 legacy=0x7f1f5415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.135+0000 7f1f4bfff640 1 -- 10.20.193.18:0/3369297776 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5415d9d0 con 0x7f1f54073530 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f82137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 10 monclient(hunting): handle_auth_done global_id 4466 payload 274 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 1 -- 10.20.193.18:0/3771736129 >> v1:10.20.193.110:6789/0 conn(0x7f2b200c9b00 legacy=0x7f2b200c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 1 -- 10.20.193.18:0/3771736129 >> v1:10.20.193.32:6789/0 conn(0x7f2b200ceca0 legacy=0x7f2b200d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.136+0000 7f2b25399640 1 -- 10.20.193.18:0/3771736129 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b200c9020 con 0x7f2b2015f650 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 10 monclient(hunting): handle_auth_done global_id 4469 payload 824 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 -- 10.20.193.18:0/1025406548 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f821415d730 msgr2=0x7f82140c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f821415d730 0x7f82140c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 -- 10.20.193.18:0/1025406548 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82141527f0 msgr2=0x7f821414e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82141527f0 0x7f821414e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.138+0000 7f82137fe640 1 -- 10.20.193.18:0/1025406548 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82141567f0 con 0x7f821415d2c0 2026-02-24T18:35:34.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.139+0000 7f1f4bfff640 1 --2- 10.20.193.18:0/3369297776 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f54073530 0x7f1f5415f3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098a0 tx=0x7f1f3c02ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c75d940083ceb58 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.140+0000 7f2b25399640 1 --2- 10.20.193.18:0/3771736129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b2015f650 0x7f2b2015fa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098a0 tx=0x7f2b1002ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af36f2df442d5406 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe688062470 con 0x7fe6a0155ef0 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7fe677fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7fe677fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7f82137fe640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f820402ef10 tx=0x7f82040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7f8210ff9640 10 client.?.objecter ms_handle_connect 0x7f821415d2c0 2026-02-24T18:35:34.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.141+0000 7f8210ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.142+0000 7f9b5dd74640 1 --2- 10.20.193.18:0/2874150314 >> v2:10.20.193.32:3300/0 conn(0x7f9b6015d2c0 0x7f9b6015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.143+0000 7f9b5dd74640 10 monclient(hunting): get_auth_request con 0x7f9b6015d2c0 auth_method 0 2026-02-24T18:35:34.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.143+0000 7f9b5dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.143+0000 7f9b5dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.143+0000 7f9b5dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f2e7db49640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_done global_id 4465 payload 274 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 1 -- 10.20.193.18:0/2874150314 >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 msgr2=0x7f9b60073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 1 --2- 10.20.193.18:0/2874150314 >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 0x7f9b60073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 1 -- 10.20.193.18:0/2874150314 >> v1:10.20.193.18:6789/0 conn(0x7f9b6015d730 legacy=0x7f9b600ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.144+0000 7f9b5dd74640 1 -- 10.20.193.18:0/2874150314 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b600741e0 con 0x7f9b6015d2c0 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53280/0 (socket says 10.20.193.18:53280) 2026-02-24T18:35:34.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 1 -- 10.20.193.18:0/681111970 learned_addr learned my addr 10.20.193.18:0/681111970 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 10 monclient(hunting): get_auth_request con 0x7f2e7815f900 auth_method 0 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.145+0000 7f2e7db49640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.146+0000 7f2e7d348640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2e78073530 0x7f2e781f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.146+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.146+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.146+0000 7f2e7db49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.146+0000 7f9b5dd74640 1 --2- 10.20.193.18:0/2874150314 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d2c0 0x7f9b6015f3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b6008f510 tx=0x7f9b4802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a3a631275bc7f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): get_auth_request con 0x7f2e78073530 auth_method 0 2026-02-24T18:35:34.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f90ee50c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f90e80c6190 0x7f90e80c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f2e7d348640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.147+0000 7f1849dd5640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1844152bb0 0x7f18440c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f90e80c6190 0x7f90e80c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44344/0 (socket says 10.20.193.18:44344) 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 1 -- 10.20.193.18:0/3730105012 learned_addr learned my addr 10.20.193.18:0/3730105012 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 10 monclient(hunting): get_auth_request con 0x7f90e80c6190 auth_method 0 2026-02-24T18:35:34.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.148+0000 7f90ee50c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 10 monclient(hunting): handle_auth_done global_id 4468 payload 824 2026-02-24T18:35:34.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 -- 10.20.193.18:0/681111970 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2e78073530 msgr2=0x7f2e781f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2e78073530 0x7f2e781f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 -- 10.20.193.18:0/681111970 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e781527f0 msgr2=0x7f2e781f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e781527f0 0x7f2e781f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.149+0000 7f2e7db49640 1 -- 10.20.193.18:0/681111970 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e781f17c0 con 0x7f2e7815f900 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1844152bb0 0x7f18440c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44328/0 (socket says 10.20.193.18:44328) 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 1 -- 10.20.193.18:0/1778995140 learned_addr learned my addr 10.20.193.18:0/1778995140 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 10 monclient(hunting): get_auth_request con 0x7f1844152bb0 auth_method 0 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.150+0000 7f1849dd5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f90ee50c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f90ee50c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f90ee50c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f1849dd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f1849dd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7f1849dd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.151+0000 7fc88ffff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8901590a0 0x7fc8901f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7fc88f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 10 monclient(hunting): handle_auth_done global_id 4475 payload 274 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 1 -- 10.20.193.18:0/3730105012 >> v2:10.20.193.32:3300/0 conn(0x7f90e81527d0 msgr2=0x7f90e806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 1 --2- 10.20.193.18:0/3730105012 >> v2:10.20.193.32:3300/0 conn(0x7f90e81527d0 0x7f90e806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 1 -- 10.20.193.18:0/3730105012 >> v1:10.20.193.110:6789/0 conn(0x7f90e8152bb0 legacy=0x7f90e80c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.152+0000 7f90ee50c640 1 -- 10.20.193.18:0/3730105012 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e806e8c0 con 0x7f90e80c6190 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53286/0 (socket says 10.20.193.18:53286) 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 1 -- 10.20.193.18:0/405818648 learned_addr learned my addr 10.20.193.18:0/405818648 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 10 monclient(hunting): get_auth_request con 0x7fc890072480 auth_method 0 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7fc88f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 10 monclient(hunting): handle_auth_done global_id 4478 payload 274 2026-02-24T18:35:34.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 1 -- 10.20.193.18:0/1778995140 >> v1:10.20.193.32:6789/0 conn(0x7f18440c5fc0 legacy=0x7f18440c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 1 -- 10.20.193.18:0/1778995140 >> v1:10.20.193.18:6789/0 conn(0x7f18441527d0 legacy=0x7f184406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.153+0000 7f1849dd5640 1 -- 10.20.193.18:0/1778995140 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184406e8c0 con 0x7f1844152bb0 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88ffff640 10 monclient(hunting): get_auth_request con 0x7fc8901590a0 auth_method 0 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.154+0000 7fc88ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.155+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.155+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.155+0000 7fc88ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7f90ee50c640 1 --2- 10.20.193.18:0/3730105012 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e80c8e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90e8158940 tx=0x7f90e402f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344a0ed282feb948 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 10 monclient(hunting): handle_auth_done global_id 4471 payload 824 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 -- 10.20.193.18:0/405818648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8901590a0 msgr2=0x7fc8901f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8901590a0 0x7fc8901f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 -- 10.20.193.18:0/405818648 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 msgr2=0x7fc8901f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8901f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7fc88f7fe640 1 -- 10.20.193.18:0/405818648 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8900c37c0 con 0x7fc890072480 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7f2e7db49640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e74039040 tx=0x7f2e74004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7f2e65ffb640 10 client.?.objecter ms_handle_connect 0x7f2e7815f900 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.156+0000 7f2e65ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.157+0000 7f1849dd5640 1 --2- 10.20.193.18:0/1778995140 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18440c5a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1830009970 tx=0x7f183002eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f761b620829c116 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.159+0000 7fc88f7fe640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00b500 tx=0x7fc87c00b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.159+0000 7fc86bfff640 10 client.?.objecter ms_handle_connect 0x7fc890072480 2026-02-24T18:35:34.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.159+0000 7fc86bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74046020 con 0x7f2e7815f900 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: dump: 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74037dc0 con 0x7f2e7815f900 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74041670 con 0x7f2e7815f900 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e7f5d3640 5 monclient: authenticate success, global_id 4468 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e7f5d3640 10 monclient: _renew_subs 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e7f5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e781f1aa0 con 0x7f2e7815f900 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e677fe640 4 set_mon_vals no callback set 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.161+0000 7f2e65ffb640 10 monclient: dump: 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 client.4468.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 monclient: _renew_subs 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781f2060 con 0x7f2e7815f900 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 1 librados: init done 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.162+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e780747a0 con 0x7f2e7815f900 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 1 -- 10.20.193.18:0/3771736129 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1003c070 con 0x7f2b2015f650 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: dump: 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.166 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 1 -- 10.20.193.18:0/3771736129 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1002fa40 con 0x7f2b2015f650 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 1 -- 10.20.193.18:0/3771736129 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1002fe10 con 0x7f2b2015f650 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b27624640 5 monclient: authenticate success, global_id 4466 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b27624640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b27624640 4 set_mon_vals no callback set 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.163+0000 7f2b0ffff640 10 monclient: dump: 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 10 monclient: shutdown 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- 10.20.193.18:0/3771736129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b2015f650 msgr2=0x7f2b2015fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 --2- 10.20.193.18:0/3771736129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b2015f650 0x7f2b2015fa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098a0 tx=0x7f2b1002ed80 comp rx=0 tx=0).stop 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- 10.20.193.18:0/3771736129 shutdown_connections 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 --2- 10.20.193.18:0/3771736129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b2015f650 0x7f2b2015fa30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- 10.20.193.18:0/3771736129 >> 10.20.193.18:0/3771736129 conn(0x7f2b20072fc0 msgr2=0x7f2b200733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- 10.20.193.18:0/3771736129 shutdown_connections 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- 10.20.193.18:0/3771736129 wait complete. 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 10 monclient: monmap: 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 librados: starting msgr at 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 librados: starting objecter 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 Processor -- start 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 -- start start 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 librados: setting wanted keys 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 1 librados: calling monclient init 2026-02-24T18:35:34.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.164+0000 7f2b27624640 10 monclient: init 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 1 -- 10.20.193.18:0/2582020374 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc02fcd0 con 0x7f5be815f900 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: dump: 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 1 -- 10.20.193.18:0/2582020374 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0373f0 con 0x7f5be815f900 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 5 monclient: authenticate success, global_id 4460 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 1 -- 10.20.193.18:0/2582020374 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0377c0 con 0x7f5be815f900 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 4 set_mon_vals no callback set 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5be67fc640 10 monclient: dump: 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.170 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b200c9b00 0x7f2b200d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-c con 0x7f2b200c9b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 0x7f2b201f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-a con 0x7f2b200ceca0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 0x7f2b201f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): picked mon.noname-b con 0x7f2b2015f650 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200cf0f0 con 0x7f2b200ceca0 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200c95e0 con 0x7f2b2015f650 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201f77a0 con 0x7f2b200c9b00 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f2b27624640 10 monclient(hunting): authenticate will time out at 1477.846441s 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 10 monclient: shutdown 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 -- 10.20.193.18:0/2582020374 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 msgr2=0x7f5be80ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2582020374 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be80ccf70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5be806cdf0 tx=0x7f5bdc02f270 comp rx=0 tx=0).stop 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 -- 10.20.193.18:0/2582020374 shutdown_connections 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2582020374 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be80ccf70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2582020374 >> v2:10.20.193.18:3300/0 conn(0x7f5be8073530 0x7f5be815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 -- 10.20.193.18:0/2582020374 >> 10.20.193.18:0/2582020374 conn(0x7f5be80ce730 msgr2=0x7f5be80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 -- 10.20.193.18:0/2582020374 shutdown_connections 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.165+0000 7f5bede0b640 1 -- 10.20.193.18:0/2582020374 wait complete. 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204044020 con 0x7f821415d2c0 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: dump: 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8204037440 con 0x7f821415d2c0 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820403c040 con 0x7f821415d2c0 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f8210ff9640 10 monclient: dump: 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f82127fc640 4 set_mon_vals no callback set 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 5 monclient: authenticate success, global_id 4469 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _renew_subs 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8214156ad0 con 0x7f821415d2c0 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 client.4469.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _renew_subs 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8214157090 con 0x7f821415d2c0 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 1 librados: init done 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 10 monclient: monmap: 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 librados: starting msgr at 2026-02-24T18:35:34.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 librados: starting objecter 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 Processor -- start 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 1 -- 10.20.193.18:0/2143703699 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c03c070 con 0x7f4518152b30 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: dump: 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 1 -- 10.20.193.18:0/2143703699 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c02fc10 con 0x7f4518152b30 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 1 -- 10.20.193.18:0/2143703699 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c037440 con 0x7f4518152b30 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f451d911640 5 monclient: authenticate success, global_id 4457 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f451d911640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f451d911640 4 set_mon_vals no callback set 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f4515ffb640 10 monclient: dump: 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 -- start start 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 librados: setting wanted keys 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 1 librados: calling monclient init 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f5bede0b640 10 monclient: init 2026-02-24T18:35:34.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.166+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8214152bd0 con 0x7f821415d2c0 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 1 -- 10.20.193.18:0/3369297776 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c03c070 con 0x7f1f54073530 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: dump: 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 1 -- 10.20.193.18:0/3369297776 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3c02fad0 con 0x7f1f54073530 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 1 -- 10.20.193.18:0/3369297776 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c037440 con 0x7f1f54073530 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 5 monclient: authenticate success, global_id 4463 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 4 set_mon_vals no callback set 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 10 monclient: shutdown 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 -- 10.20.193.18:0/2143703699 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4518152b30 msgr2=0x7f451815b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 --2- 10.20.193.18:0/2143703699 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4518152b30 0x7f451815b340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f450c0099a0 tx=0x7f450c02efe0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f4b7fe640 10 monclient: dump: 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.178 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-a con 0x7f5be8073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5be81527f0 0x7f5be8145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-b con 0x7f5be81527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be81e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): picked mon.noname-c con 0x7f5be815f900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80741e0 con 0x7f5be8073530 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8073f00 con 0x7f5be81527f0 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80744c0 con 0x7f5be815f900 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f5bede0b640 10 monclient(hunting): authenticate will time out at 1477.848234s 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 -- 10.20.193.18:0/2143703699 shutdown_connections 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 --2- 10.20.193.18:0/2143703699 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4518152b30 0x7f451815b340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 -- 10.20.193.18:0/2143703699 >> 10.20.193.18:0/2143703699 conn(0x7f4518074080 msgr2=0x7f4518074490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 -- 10.20.193.18:0/2143703699 shutdown_connections 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 -- 10.20.193.18:0/2143703699 wait complete. 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 10 monclient: shutdown 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 -- 10.20.193.18:0/3369297776 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f54073530 msgr2=0x7f1f5415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/3369297776 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f54073530 0x7f1f5415f3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098a0 tx=0x7f1f3c02ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 -- 10.20.193.18:0/3369297776 shutdown_connections 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/3369297776 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f54073530 0x7f1f5415f3c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 -- 10.20.193.18:0/3369297776 >> 10.20.193.18:0/3369297776 conn(0x7f1f540ce730 msgr2=0x7f1f540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 -- 10.20.193.18:0/3369297776 shutdown_connections 2026-02-24T18:35:34.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f1f5ae58640 1 -- 10.20.193.18:0/3369297776 wait complete. 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 10 monclient: monmap: 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 librados: starting msgr at 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.167+0000 7f451d911640 1 librados: starting objecter 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 Processor -- start 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 -- start start 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 librados: setting wanted keys 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 librados: calling monclient init 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient: init 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f2b25399640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b200c9b00 0x7f2b200d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 10 monclient: monmap: 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 librados: starting msgr at 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 librados: starting objecter 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 Processor -- start 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): picked mon.noname-a con 0x7f4518152b30 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4518152f10 0x7f4518136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): picked mon.noname-b con 0x7f4518152f10 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f451815c730 0x7f45180c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): picked mon.noname-c con 0x7f451815c730 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f451815b880 con 0x7f4518152b30 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f451815ae20 con 0x7f4518152f10 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f451815ba00 con 0x7f451815c730 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f451d911640 10 monclient(hunting): authenticate will time out at 1477.849619s 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 -- start start 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 librados: setting wanted keys 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 1 librados: calling monclient init 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.168+0000 7f1f5ae58640 10 monclient: init 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1f54073530 0x7f1f540c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-b con 0x7f1f54073530 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f541527f0 0x7f1f541578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-c con 0x7f1f541527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): picked mon.noname-a con 0x7f1f5415f900 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540741e0 con 0x7f1f5415f900 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54073f00 con 0x7f1f54073530 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540744c0 con 0x7f1f541527f0 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.169+0000 7f1f5ae58640 10 monclient(hunting): authenticate will time out at 1477.850141s 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.173+0000 7f2b25b9a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 0x7f2b201f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.174+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2e74041810 con 0x7f2e7815f900 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.175+0000 7f2e65ffb640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 0x7f2e4807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.175+0000 7f2e7d348640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 0x7f2e4807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.175+0000 7f2e7d348640 10 monclient: get_auth_request con 0x7f2e48078e00 auth_method 0 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.175+0000 7f2e65ffb640 10 client.4468.objecter ms_handle_connect 0x7f2e48078e00 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.175+0000 7f2e7d348640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 0x7f2e4807b2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004660 tx=0x7f2e68009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.176+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f2e74083a50 con 0x7f2e7815f900 2026-02-24T18:35:34.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.176+0000 7f2e65ffb640 10 client.4468.objecter ms_dispatch2 0x7f2e78000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.176+0000 7f2e65ffb640 3 client.4468.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.176+0000 7f2e65ffb640 3 client.4468.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2e65ffb640 20 client.4468.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 1 -- 10.20.193.18:0/2874150314 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4803c070 con 0x7f9b6015d2c0 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: dump: 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 1 -- 10.20.193.18:0/2874150314 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4802fa50 con 0x7f9b6015d2c0 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 0x7f2b201f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53314/0 (socket says 10.20.193.18:53314) 2026-02-24T18:35:34.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 1 -- 10.20.193.18:0/1451771671 learned_addr learned my addr 10.20.193.18:0/1451771671 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 1 -- 10.20.193.18:0/2874150314 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4802fe20 con 0x7f9b6015d2c0 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b64bb3640 5 monclient: authenticate success, global_id 4465 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 10 monclient(hunting): get_auth_request con 0x7f2b2015f650 auth_method 0 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f2b25b9a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b64bb3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b64bb3640 4 set_mon_vals no callback set 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.177+0000 7f9b5d573640 10 monclient: dump: 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 10 monclient: shutdown 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 -- 10.20.193.18:0/2874150314 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d2c0 msgr2=0x7f9b6015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/2874150314 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d2c0 0x7f9b6015f3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b6008f510 tx=0x7f9b4802ede0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 -- 10.20.193.18:0/2874150314 shutdown_connections 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/2874150314 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d2c0 0x7f9b6015f3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/2874150314 >> v2:10.20.193.18:3300/0 conn(0x7f9b601527f0 0x7f9b60073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 -- 10.20.193.18:0/2874150314 >> 10.20.193.18:0/2874150314 conn(0x7f9b600ce730 msgr2=0x7f9b600ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c004480 con 0x7fc890072480 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: dump: 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c014070 con 0x7fc890072480 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c002920 con 0x7fc890072480 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc86bfff640 10 monclient: dump: 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.188 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc88dffb640 4 set_mon_vals no callback set 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 5 monclient: authenticate success, global_id 4471 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 -- 10.20.193.18:0/2874150314 shutdown_connections 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _renew_subs 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8900c3aa0 con 0x7fc890072480 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 1 -- 10.20.193.18:0/2874150314 wait complete. 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 client.4471.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _renew_subs 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8900c4060 con 0x7fc890072480 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 1 librados: init done 2026-02-24T18:35:34.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f2b25b9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f2b25b9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f2b25b9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7f9b64bb3640 10 monclient: monmap: 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.178+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc890158730 con 0x7fc890072480 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_done global_id 4512 payload 274 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 1 -- 10.20.193.18:0/480914891 >> v1:10.20.193.32:6789/0 conn(0x7f5a38073530 legacy=0x7f5a3815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 1 -- 10.20.193.18:0/480914891 >> v1:10.20.193.18:6789/0 conn(0x7f5a381527f0 legacy=0x7f5a3815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 librados: starting msgr at 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 librados: starting objecter 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f5a3da6b640 1 -- 10.20.193.18:0/480914891 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a38073c20 con 0x7f5a3815fad0 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 Processor -- start 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 -- start start 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 librados: setting wanted keys 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 librados: calling monclient init 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient: init 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 0x7f9b60156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-c con 0x7f9b601527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 0x7f9b60157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-a con 0x7f9b6015d2c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d730 0x7f9b60158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): picked mon.noname-b con 0x7f9b6015d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60074fa0 con 0x7f9b6015d2c0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600747a0 con 0x7f9b6015d730 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60075120 con 0x7f9b601527f0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.179+0000 7f9b64bb3640 10 monclient(hunting): authenticate will time out at 1477.860887s 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.189+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1ddc041c80 con 0x7f1de81527f0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.189+0000 7f1dceffd640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 0x7f1db407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.189+0000 7f1de5d74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 0x7f1db407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.189+0000 7f1de5d74640 10 monclient: get_auth_request con 0x7f1db4078e50 auth_method 0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.190+0000 7f1de5d74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 0x7f1db407b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1de815efc0 tx=0x7f1dd0006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.190+0000 7f1dceffd640 10 client.4503.objecter ms_handle_connect 0x7f1db4078e50 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.197+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/480914891 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a3815fad0 0x7f5a380ccf70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009d90 tx=0x7f5a3402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf6cd7cc0957dd server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec03c070 con 0x7efbf0004900 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: dump: 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbec02f770 con 0x7efbf0004900 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 1 -- 10.20.193.18:0/2517153324 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec009550 con 0x7efbf0004900 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 4 set_mon_vals no callback set 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efbfb7fe640 10 monclient: dump: 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 10 monclient: shutdown 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 -- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 msgr2=0x7efbf0004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 0x7efbf0004d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efbec009980 tx=0x7efbec02ed10 comp rx=0 tx=0).stop 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 -- 10.20.193.18:0/2517153324 shutdown_connections 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/2517153324 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efbf0004900 0x7efbf0004d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 -- 10.20.193.18:0/2517153324 >> 10.20.193.18:0/2517153324 conn(0x7efc040ce730 msgr2=0x7efc040ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 -- 10.20.193.18:0/2517153324 shutdown_connections 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.198+0000 7efc0b1dc640 1 -- 10.20.193.18:0/2517153324 wait complete. 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 10 monclient: monmap: 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.200 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 librados: starting msgr at 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 librados: starting objecter 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 Processor -- start 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 -- start start 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 librados: setting wanted keys 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 1 librados: calling monclient init 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.199+0000 7efc0b1dc640 10 monclient: init 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0004900 0x7efc040c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-c con 0x7efbf0004900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc040ca390 0x7efc041f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-b con 0x7efc040ca390 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): picked mon.noname-a con 0x7efc041f5730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efc040741e0 con 0x7efc041f5730 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc04073f00 con 0x7efc040ca390 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc040744c0 con 0x7efbf0004900 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.200+0000 7efc0b1dc640 10 monclient(hunting): authenticate will time out at 1477.881155s 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f1ddc02fa20 con 0x7f1de81527f0 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 10 client.4503.objecter ms_dispatch2 0x7f1de8000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 3 client.4503.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 3 client.4503.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 20 client.4503.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1ddc087ec0 con 0x7f1de81527f0 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.204+0000 7f1dceffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.208+0000 7f3f23353640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:34.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.208+0000 7f3f23353640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:34.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.208+0000 7f3f23353640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.208+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f1c0cc1b0 con 0x7f3f1c15f900 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.214+0000 7f5be77fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41540/0 (socket says 10.20.193.18:41540) 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 1 -- 10.20.193.18:0/2446358923 learned_addr learned my addr 10.20.193.18:0/2446358923 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 10 monclient(hunting): get_auth_request con 0x7f5be8073530 auth_method 0 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f5be77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f2b24b98640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 0x7f2b201f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f2b24b98640 10 monclient(hunting): get_auth_request con 0x7f2b200ceca0 auth_method 0 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f2b24b98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f2b24b98640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.215+0000 7f2b24b98640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f5be77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f5be77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f5be77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f2b24b98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f2b24b98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.216+0000 7f2b24b98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 -- 10.20.193.18:0/1451771671 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b200c9b00 msgr2=0x7f2b200d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b200c9b00 0x7f2b200d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 -- 10.20.193.18:0/1451771671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 msgr2=0x7f2b201f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 0x7f2b201f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f2b24b98640 1 -- 10.20.193.18:0/1451771671 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201f7a80 con 0x7f2b200ceca0 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 10 monclient(hunting): handle_auth_done global_id 4515 payload 824 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 -- 10.20.193.18:0/2446358923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 msgr2=0x7f5be81e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be81e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 -- 10.20.193.18:0/2446358923 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5be81527f0 msgr2=0x7f5be8145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5be81527f0 0x7f5be8145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.220+0000 7f5be77fe640 1 -- 10.20.193.18:0/2446358923 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80c9df0 con 0x7f5be8073530 2026-02-24T18:35:34.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f2b24b98640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 0x7f2b201f4590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c460 tx=0x7f2b1400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f2b0dffb640 10 client.?.objecter ms_handle_connect 0x7f2b200ceca0 2026-02-24T18:35:34.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f2b0dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f5be77fe640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b500 tx=0x7f5bd400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f5bc3fff640 10 client.?.objecter ms_handle_connect 0x7f5be8073530 2026-02-24T18:35:34.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.224+0000 7f5bc3fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680052d0 con 0x7f0878073530 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: dump: 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868005470 con 0x7f0878073530 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680038d0 con 0x7f0878073530 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 5 monclient: authenticate success, global_id 4506 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 10 monclient: _renew_subs 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086effd640 4 set_mon_vals no callback set 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781f4490 con 0x7f0878073530 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f086d7fa640 10 monclient: dump: 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 10 client.4506.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 10 monclient: _renew_subs 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08780c7410 con 0x7f0878073530 2026-02-24T18:35:34.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.225+0000 7f087ecf5640 1 librados: init done 2026-02-24T18:35:34.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f087ecf5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f087ecf5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f087ecf5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0878160030 con 0x7f0878073530 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 1 -- 10.20.193.18:0/480914891 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3403c070 con 0x7f5a3815fad0 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: dump: 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 1 -- 10.20.193.18:0/480914891 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34037440 con 0x7f5a3815fad0 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 1 -- 10.20.193.18:0/480914891 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037810 con 0x7f5a3815fad0 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a3f4f5640 5 monclient: authenticate success, global_id 4512 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a3f4f5640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a3f4f5640 4 set_mon_vals no callback set 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.226+0000 7f5a27fff640 10 monclient: dump: 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 10 monclient: shutdown 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/480914891 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a3815fad0 msgr2=0x7f5a380ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/480914891 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a3815fad0 0x7f5a380ccf70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009d90 tx=0x7f5a3402f230 comp rx=0 tx=0).stop 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/480914891 shutdown_connections 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/480914891 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a3815fad0 0x7f5a380ccf70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/480914891 >> 10.20.193.18:0/480914891 conn(0x7f5a380ce730 msgr2=0x7f5a380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/480914891 shutdown_connections 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/480914891 wait complete. 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 10 monclient: monmap: 2026-02-24T18:35:34.231 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 librados: starting msgr at 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 librados: starting objecter 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 Processor -- start 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 -- start start 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 librados: setting wanted keys 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 1 librados: calling monclient init 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.227+0000 7f5a3f4f5640 10 monclient: init 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a38073530 0x7f5a38149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-a con 0x7f5a38073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 0x7f5a38159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-c con 0x7f5a381527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 0x7f5a381f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): picked mon.noname-b con 0x7f5a3815fad0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380744c0 con 0x7f5a38073530 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380741e0 con 0x7f5a3815fad0 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380747a0 con 0x7f5a381527f0 2026-02-24T18:35:34.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f5a3f4f5640 10 monclient(hunting): authenticate will time out at 1477.909345s 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7efc09752640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f4516ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.228+0000 7f1f593ce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41562/0 (socket says 10.20.193.18:41562) 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 1 -- 10.20.193.18:0/805153488 learned_addr learned my addr 10.20.193.18:0/805153488 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 10 monclient(hunting): get_auth_request con 0x7efc041f5730 auth_method 0 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7efc09752640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41550/0 (socket says 10.20.193.18:41550) 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 1 -- 10.20.193.18:0/4094743582 learned_addr learned my addr 10.20.193.18:0/4094743582 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): get_auth_request con 0x7f4518152b30 auth_method 0 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41554/0 (socket says 10.20.193.18:41554) 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 1 -- 10.20.193.18:0/2826628869 learned_addr learned my addr 10.20.193.18:0/2826628869 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 10 monclient(hunting): get_auth_request con 0x7f1f5415f900 auth_method 0 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f1f593ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.229+0000 7f4516ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7f1f593ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7f1f593ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7f1f593ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7efc09752640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7efc09752640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.230+0000 7efc09752640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 10 monclient(hunting): handle_auth_done global_id 4521 payload 824 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 1 -- 10.20.193.18:0/4094743582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f451815c730 msgr2=0x7f45180c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f451815c730 0x7f45180c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 1 -- 10.20.193.18:0/4094743582 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4518152f10 msgr2=0x7f4518136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4518152f10 0x7f4518136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.231+0000 7f4516ffd640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.232+0000 7f4516ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.232+0000 7f4516ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.232+0000 7f4516ffd640 1 -- 10.20.193.18:0/4094743582 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45181560a0 con 0x7f4518152b30 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.233+0000 7fe6a5832640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.233+0000 7fe6a5832640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.233+0000 7fe6a5832640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.233+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe6a00ca690 con 0x7fe6a0155ef0 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 10 monclient(hunting): handle_auth_done global_id 4524 payload 824 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 -- 10.20.193.18:0/805153488 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0004900 msgr2=0x7efc040c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0004900 0x7efc040c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 -- 10.20.193.18:0/805153488 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc040ca390 msgr2=0x7efc041f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc040ca390 0x7efc041f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.234+0000 7efc09752640 1 -- 10.20.193.18:0/805153488 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc041f5bc0 con 0x7efc041f5730 2026-02-24T18:35:34.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 -- 10.20.193.18:0/2826628869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f541527f0 msgr2=0x7f1f541578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f541527f0 0x7f1f541578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 -- 10.20.193.18:0/2826628869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1f54073530 msgr2=0x7f1f540c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1f54073530 0x7f1f540c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 monclient: found mon.noname-a 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.235+0000 7f1f593ce640 1 -- 10.20.193.18:0/2826628869 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f540c58d0 con 0x7f1f5415f900 2026-02-24T18:35:34.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f1f593ce640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b990 tx=0x7f1f5000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f1f497fa640 10 client.?.objecter ms_handle_connect 0x7f1f5415f900 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f1f497fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f4516ffd640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f450c009ad0 tx=0x7f450c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f44effff640 10 client.?.objecter ms_handle_connect 0x7f4518152b30 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7f44effff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7efc09752640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc040ce270 tx=0x7efc0000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7efbf97fa640 10 client.?.objecter ms_handle_connect 0x7efc041f5730 2026-02-24T18:35:34.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.237+0000 7efbf97fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.240+0000 7f9b5dd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 0x7f9b60157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 0x7f9b60157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41560/0 (socket says 10.20.193.18:41560) 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 1 -- 10.20.193.18:0/3774607672 learned_addr learned my addr 10.20.193.18:0/3774607672 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 10 monclient(hunting): get_auth_request con 0x7f9b6015d2c0 auth_method 0 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.242+0000 7f9b5dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.243+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.243+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.243+0000 7f9b5dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 1 -- 10.20.193.18:0/3730105012 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e402fcd0 con 0x7f90e80c6190 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: dump: 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 1 -- 10.20.193.18:0/3730105012 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e40373f0 con 0x7f90e80c6190 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 1 -- 10.20.193.18:0/3730105012 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e40377c0 con 0x7f90e80c6190 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90eff96640 5 monclient: authenticate success, global_id 4475 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90eff96640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90eff96640 4 set_mon_vals no callback set 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.266+0000 7f90ecd09640 10 monclient: dump: 2026-02-24T18:35:34.268 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.269 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 10 monclient: shutdown 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- 10.20.193.18:0/3730105012 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 msgr2=0x7f90e80c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 --2- 10.20.193.18:0/3730105012 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e80c8e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90e8158940 tx=0x7f90e402f270 comp rx=0 tx=0).stop 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- 10.20.193.18:0/3730105012 shutdown_connections 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 --2- 10.20.193.18:0/3730105012 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e80c8e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 --2- 10.20.193.18:0/3730105012 >> v2:10.20.193.32:3300/0 conn(0x7f90e81527d0 0x7f90e806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- 10.20.193.18:0/3730105012 >> 10.20.193.18:0/3730105012 conn(0x7f90e80cd780 msgr2=0x7f90e80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- 10.20.193.18:0/3730105012 shutdown_connections 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- 10.20.193.18:0/3730105012 wait complete. 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 10 monclient: monmap: 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.270 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 librados: starting msgr at 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 librados: starting objecter 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 Processor -- start 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 -- start start 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 librados: setting wanted keys 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 1 librados: calling monclient init 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.267+0000 7f90eff96640 10 monclient: init 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-c con 0x7f90e80c6190 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f90e81527d0 0x7f90e81531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-b con 0x7f90e81527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f90e8152bb0 0x7f90e8157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): picked mon.noname-a con 0x7f90e8152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f90e806dfa0 con 0x7f90e8152bb0 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90e806ee80 con 0x7f90e81527d0 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90e80c93a0 con 0x7f90e80c6190 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.268+0000 7f90eff96640 10 monclient(hunting): authenticate will time out at 1477.949391s 2026-02-24T18:35:34.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 1 -- 10.20.193.18:0/1778995140 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183003c070 con 0x7f1844152bb0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: dump: 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 1 -- 10.20.193.18:0/1778995140 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183002fb60 con 0x7f1844152bb0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 1 -- 10.20.193.18:0/1778995140 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830037440 con 0x7f1844152bb0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 5 monclient: authenticate success, global_id 4478 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 4 set_mon_vals no callback set 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f18495d4640 10 monclient: dump: 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.273 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 10 monclient: shutdown 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 -- 10.20.193.18:0/1778995140 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 msgr2=0x7f18440c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 --2- 10.20.193.18:0/1778995140 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18440c5a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1830009970 tx=0x7f183002eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 -- 10.20.193.18:0/1778995140 shutdown_connections 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 --2- 10.20.193.18:0/1778995140 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18440c5a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 -- 10.20.193.18:0/1778995140 >> 10.20.193.18:0/1778995140 conn(0x7f18440cd780 msgr2=0x7f18440cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 -- 10.20.193.18:0/1778995140 shutdown_connections 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.269+0000 7f184c861640 1 -- 10.20.193.18:0/1778995140 wait complete. 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 10 monclient: build_initial_monmap 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 10 monclient: monmap: 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:34.274 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 librados: starting msgr at 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 librados: starting objecter 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 Processor -- start 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 -- start start 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 librados: setting wanted keys 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 1 librados: calling monclient init 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.270+0000 7f184c861640 10 monclient: init 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f18440c5fc0 0x7f1844073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): picked mon.noname-a con 0x7f18440c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f18441527d0 0x7f18441f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): picked mon.noname-b con 0x7f18441527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): picked mon.noname-c con 0x7f1844152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f184406ee80 con 0x7f18440c5fc0 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18440c6470 con 0x7f18441527d0 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18440c9570 con 0x7f1844152bb0 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:34.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.271+0000 7f184c861640 10 monclient(hunting): authenticate will time out at 1477.952110s 2026-02-24T18:35:34.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.277+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f820403f070 con 0x7f821415d2c0 2026-02-24T18:35:34.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.278+0000 7f8210ff9640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 0x7f81e407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.278+0000 7f8213fff640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 0x7f81e407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.278+0000 7f8213fff640 10 monclient: get_auth_request con 0x7f81e4078e80 auth_method 0 2026-02-24T18:35:34.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.279+0000 7f8210ff9640 10 client.4469.objecter ms_handle_connect 0x7f81e4078e80 2026-02-24T18:35:34.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.279+0000 7f8213fff640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 0x7f81e407b340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82000098a0 tx=0x7f8200006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.281+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f820402fa40 con 0x7f821415d2c0 2026-02-24T18:35:34.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.281+0000 7f8210ff9640 10 client.4469.objecter ms_dispatch2 0x7f8214000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.281+0000 7f8210ff9640 3 client.4469.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.281+0000 7f8210ff9640 3 client.4469.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.281+0000 7f8210ff9640 20 client.4469.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.282+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2e74050030 con 0x7f2e7815f900 2026-02-24T18:35:34.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.282+0000 7f2e65ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.282+0000 7f2e65ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.289+0000 7f1deca74640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.289+0000 7f1deca74640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.289+0000 7f1deca74640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.289+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f1de815e9f0 con 0x7f1de81527f0 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.290+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f820404e030 con 0x7f821415d2c0 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.290+0000 7f8210ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.290+0000 7f8210ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.293+0000 7f3f09ffb640 1 -- 10.20.193.18:0/3676213211 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f1c0739a0 con 0x7f3f1c15f900 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.293+0000 7f3f09ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.293+0000 7f3f09ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 0x7f5a38159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 0x7f5a38159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44410/0 (socket says 10.20.193.18:44410) 2026-02-24T18:35:34.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 1 -- 10.20.193.18:0/2876715624 learned_addr learned my addr 10.20.193.18:0/2876715624 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 10 monclient(hunting): get_auth_request con 0x7f5a381527f0 auth_method 0 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.294+0000 7f5a3ca69640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.295+0000 7f5a3ca69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.295+0000 7f5a3ca69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.295+0000 7f5a3ca69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 10 librados: watch_flush enter 2026-02-24T18:35:34.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 10 librados: watch_flush exit 2026-02-24T18:35:34.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 20 client.4454.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 10 client.4454.objecter successfully canceled tick 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 msgr2=0x7f3eec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 0x7f3eec07b310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c004660 tx=0x7f3f0c009210 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 10 monclient: shutdown 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 msgr2=0x7f3f1c1f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c1f2070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f18035000 tx=0x7f3f18004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 shutdown_connections 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3eec078e50 0x7f3eec07b310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f1c15f900 0x7f3f1c1f2070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f1c1527f0 0x7f3f1c1f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 --2- 10.20.193.18:0/3676213211 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f1c073530 0x7f3f1c0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 >> 10.20.193.18:0/3676213211 conn(0x7f3f1c0ce730 msgr2=0x7f3f1c0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 shutdown_connections 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 -- 10.20.193.18:0/3676213211 wait complete. 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f3f23353640 1 librados: shutdown 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.296+0000 7f90edd0b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44426/0 (socket says 10.20.193.18:44426) 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 1 -- 10.20.193.18:0/2366333145 learned_addr learned my addr 10.20.193.18:0/2366333145 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): get_auth_request con 0x7f90e80c6190 auth_method 0 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f9b5e575640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 0x7f9b60156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f90edd0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f9b5e575640 10 monclient(hunting): get_auth_request con 0x7f9b601527f0 auth_method 0 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f9b5e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f9b5e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.297+0000 7f9b5e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2026-02-24T18:35:34.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 -- 10.20.193.18:0/2366333145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f90e81527d0 msgr2=0x7f90e81531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f90e81527d0 0x7f90e81531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 -- 10.20.193.18:0/2366333145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f90e8152bb0 msgr2=0x7f90e8157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f90e8152bb0 0x7f90e8157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f90edd0b640 1 -- 10.20.193.18:0/2366333145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8153b00 con 0x7f90e80c6190 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.299+0000 7f9b5e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 10 monclient(hunting): handle_auth_done global_id 4490 payload 824 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 -- 10.20.193.18:0/3774607672 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d730 msgr2=0x7f9b60158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d730 0x7f9b60158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 -- 10.20.193.18:0/3774607672 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 msgr2=0x7f9b60157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 0x7f9b60157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.301+0000 7f9b5e575640 1 -- 10.20.193.18:0/3774607672 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b601f47b0 con 0x7f9b601527f0 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.302+0000 7f90edd0b640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b7b0 tx=0x7f90d800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.302+0000 7f90d67fc640 10 client.?.objecter ms_handle_connect 0x7f90e80c6190 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.302+0000 7f90d67fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.303+0000 7f9b5e575640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 0x7f9b60156930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b54006fd0 tx=0x7f9b5400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.303+0000 7f9b46ffd640 10 client.?.objecter ms_handle_connect 0x7f9b601527f0 2026-02-24T18:35:34.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.303+0000 7f9b46ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.304+0000 7f184add7640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247811 for osd.0 2026-02-24T18:35:34.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44434/0 (socket says 10.20.193.18:44434) 2026-02-24T18:35:34.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 1 -- 10.20.193.18:0/3599788784 learned_addr learned my addr 10.20.193.18:0/3599788784 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:34.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 10 monclient(hunting): get_auth_request con 0x7f1844152bb0 auth_method 0 2026-02-24T18:35:34.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.305+0000 7f184add7640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.306+0000 7f184add7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.306+0000 7f184add7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.306+0000 7f184add7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 10 monclient(hunting): handle_auth_done global_id 4493 payload 824 2026-02-24T18:35:34.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 -- 10.20.193.18:0/3599788784 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f18441527d0 msgr2=0x7f18441f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f18441527d0 0x7f18441f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 -- 10.20.193.18:0/3599788784 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f18440c5fc0 msgr2=0x7f1844073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f18440c5fc0 0x7f1844073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 monclient: found mon.noname-c 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f184add7640 1 -- 10.20.193.18:0/3599788784 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1844154f50 con 0x7f1844152bb0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54017070 con 0x7f9b601527f0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: dump: 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b5400c940 con 0x7f9b601527f0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400cd10 con 0x7f9b601527f0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b46ffd640 10 monclient: dump: 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b5cd72640 4 set_mon_vals no callback set 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 5 monclient: authenticate success, global_id 4490 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 10 monclient: _renew_subs 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b600c5df0 con 0x7f9b601527f0 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 10 client.4490.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 10 monclient: _renew_subs 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b600c63b0 con 0x7f9b601527f0 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.308+0000 7f9b64bb3640 1 librados: init done 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.309+0000 7f9b64bb3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.309+0000 7f9b64bb3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.309+0000 7f9b64bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.309+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b60152bd0 con 0x7f9b601527f0 2026-02-24T18:35:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.313+0000 7f184add7640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f183800b9f0 tx=0x7f183800bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.313+0000 7f182effd640 10 client.?.objecter ms_handle_connect 0x7f1844152bb0 2026-02-24T18:35:34.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.313+0000 7f182effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4004480 con 0x7f5be8073530 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: dump: 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.348 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd4014070 con 0x7f5be8073530 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4002920 con 0x7f5be8073530 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 5 monclient: authenticate success, global_id 4515 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5be5ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 10 monclient: _renew_subs 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be81e7cf0 con 0x7f5be8073530 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bc3fff640 10 monclient: dump: 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 10 client.4515.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 10 monclient: _renew_subs 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be81e82b0 con 0x7f5be8073530 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.346+0000 7f5bede0b640 1 librados: init done 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.347+0000 7f5bede0b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.347+0000 7f5bede0b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.347+0000 7f5bede0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.347+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be80747a0 con 0x7f5be8073530 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000c790 con 0x7f1f5415f900 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: dump: 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.351 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50010070 con 0x7f1f5415f900 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000cbb0 con 0x7f1f5415f900 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 5 monclient: authenticate success, global_id 4527 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 10 monclient: _renew_subs 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f4affd640 4 set_mon_vals no callback set 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f540c5bb0 con 0x7f1f5415f900 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f497fa640 10 monclient: dump: 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.352 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 10 client.4527.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 10 monclient: _renew_subs 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f540c6170 con 0x7f1f5415f900 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.349+0000 7f1f5ae58640 1 librados: init done 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.350+0000 7f1f5ae58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.350+0000 7f1f5ae58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.350+0000 7f1f5ae58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.350+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f540747a0 con 0x7f1f5415f900 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0000aa90 con 0x7efc041f5730 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: dump: 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0000ac30 con 0x7efc041f5730 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00014420 con 0x7efc041f5730 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 5 monclient: authenticate success, global_id 4524 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _renew_subs 2026-02-24T18:35:34.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbfaffd640 4 set_mon_vals no callback set 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc040c6a70 con 0x7efc041f5730 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efbf97fa640 10 monclient: dump: 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 client.4524.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _renew_subs 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc040c7030 con 0x7efc041f5730 2026-02-24T18:35:34.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 1 librados: init done 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017070 con 0x7f2b200ceca0 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: dump: 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b14005de0 con 0x7f2b200ceca0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140072f0 con 0x7f2b200ceca0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 5 monclient: authenticate success, global_id 4518 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 10 monclient: _renew_subs 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0f7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201f7d60 con 0x7f2b200ceca0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b0dffb640 10 monclient: dump: 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 10 client.4518.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 10 monclient: _renew_subs 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201f8320 con 0x7f2b200ceca0 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7f2b27624640 1 librados: init done 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.351+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc040747a0 con 0x7efc041f5730 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.352+0000 7f2b27624640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.352+0000 7f2b27624640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.352+0000 7f2b27624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.352+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b200cc020 con 0x7f2b200ceca0 2026-02-24T18:35:34.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.359+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0868003a70 con 0x7f0878073530 2026-02-24T18:35:34.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.359+0000 7f086d7fa640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 0x7f084807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.359+0000 7f086ffff640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 0x7f084807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.359+0000 7f086ffff640 10 monclient: get_auth_request con 0x7f0848078e80 auth_method 0 2026-02-24T18:35:34.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.360+0000 7f086ffff640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 0x7f084807b340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0860007fd0 tx=0x7f0860007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.360+0000 7f086d7fa640 10 client.4506.objecter ms_handle_connect 0x7f0848078e80 2026-02-24T18:35:34.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f2e7f5d3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:34.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f2e7f5d3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:34.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f2e7f5d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2e7815dc40 con 0x7f2e7815f900 2026-02-24T18:35:34.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f086805e0f0 con 0x7f0878073530 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 10 client.4506.objecter ms_dispatch2 0x7f0878000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 3 client.4506.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 3 client.4506.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 20 client.4506.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0878160030 con 0x7f0878073530 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.369+0000 7f086d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.379+0000 7f821a9d3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:34.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.379+0000 7f821a9d3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:34.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.379+0000 7f821a9d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.379+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f821414edd0 con 0x7f821415d2c0 2026-02-24T18:35:34.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.390+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc87c019460 con 0x7fc890072480 2026-02-24T18:35:34.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.390+0000 7fc86bfff640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 0x7fc86007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.391+0000 7fc88effd640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 0x7fc86007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.391+0000 7fc88effd640 10 monclient: get_auth_request con 0x7fc860078e80 auth_method 0 2026-02-24T18:35:34.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.391+0000 7fc88effd640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 0x7fc86007b340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc880006fd0 tx=0x7fc880006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.391+0000 7fc86bfff640 10 client.4471.objecter ms_handle_connect 0x7fc860078e80 2026-02-24T18:35:34.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.394+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fc87c05dcf0 con 0x7fc890072480 2026-02-24T18:35:34.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.394+0000 7fc86bfff640 10 client.4471.objecter ms_dispatch2 0x7fc890000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.394+0000 7fc86bfff640 3 client.4471.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.394+0000 7fc86bfff640 3 client.4471.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.394+0000 7fc86bfff640 20 client.4471.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.400+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc87c0623b0 con 0x7fc890072480 2026-02-24T18:35:34.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.400+0000 7fc86bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.400+0000 7fc86bfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.401+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 0x7f5a381f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.402+0000 7f5a3da6b640 10 monclient(hunting): get_auth_request con 0x7f5a3815fad0 auth_method 0 2026-02-24T18:35:34.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.402+0000 7f5a3da6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:34.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.402+0000 7f5a3da6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:34.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.402+0000 7f5a3da6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:34.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:34.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:34.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:34.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f2e65ffb640 1 -- 10.20.193.18:0/681111970 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e780739a0 con 0x7f2e7815f900 2026-02-24T18:35:34.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f2e65ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:34.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.404+0000 7f2e65ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.406 INFO:teuthology.orchestra.run.trial018.stdout:98784247809 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 10 monclient(hunting): handle_auth_done global_id 4480 payload 824 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 -- 10.20.193.18:0/2876715624 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 msgr2=0x7f5a38159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 0x7f5a38159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 -- 10.20.193.18:0/2876715624 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a38073530 msgr2=0x7f5a38149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a38073530 0x7f5a38149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:34.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 monclient: found mon.noname-b 2026-02-24T18:35:34.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:34.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.406+0000 7f5a3da6b640 1 -- 10.20.193.18:0/2876715624 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3815a330 con 0x7f5a3815fad0 2026-02-24T18:35:34.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 10 librados: watch_flush enter 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 10 librados: watch_flush exit 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 20 client.4468.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 10 client.4468.objecter successfully canceled tick 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 msgr2=0x7f2e4807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 0x7f2e4807b2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004660 tx=0x7f2e68009110 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 10 monclient: shutdown 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 msgr2=0x7f2e781f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e74039040 tx=0x7f2e74004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 shutdown_connections 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2e48078e00 0x7f2e4807b2c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2e7815f900 0x7f2e781f3080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2e781527f0 0x7f2e781f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 --2- 10.20.193.18:0/681111970 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2e78073530 0x7f2e781f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 >> 10.20.193.18:0/681111970 conn(0x7f2e780ce730 msgr2=0x7f2e780cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 shutdown_connections 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 -- 10.20.193.18:0/681111970 wait complete. 2026-02-24T18:35:34.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.407+0000 7f2e7f5d3640 1 librados: shutdown 2026-02-24T18:35:34.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.408+0000 7f5a3da6b640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 0x7f5a381f4130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a34036000 tx=0x7f5a34004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.408+0000 7f5a25ffb640 10 client.?.objecter ms_handle_connect 0x7f5a3815fad0 2026-02-24T18:35:34.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.408+0000 7f5a25ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:34.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.410+0000 7fe677fff640 1 -- 10.20.193.18:0/1605784181 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe688061d70 con 0x7fe6a0155ef0 2026-02-24T18:35:34.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.410+0000 7fe677fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:34.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.410+0000 7fe677fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.411 INFO:teuthology.orchestra.run.trial018.stdout:90194313218 2026-02-24T18:35:34.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 10 librados: watch_flush enter 2026-02-24T18:35:34.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 10 librados: watch_flush exit 2026-02-24T18:35:34.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 20 client.4456.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34047020 con 0x7f5a3815fad0 2026-02-24T18:35:34.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 10 client.4456.objecter successfully canceled tick 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: dump: 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.414 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3403d040 con 0x7f5a3815fad0 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3403c050 con 0x7f5a3815fad0 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 5 monclient: authenticate success, global_id 4480 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _renew_subs 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 msgr2=0x7fe67007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 0x7fe67007b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 10 monclient: shutdown 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a277fe640 4 set_mon_vals no callback set 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3815a610 con 0x7f5a3815fad0 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 msgr2=0x7fe6a0140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b500 tx=0x7fe68800b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a25ffb640 10 monclient: dump: 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.416 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 client.4480.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _renew_subs 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3815abd0 con 0x7f5a3815fad0 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 shutdown_connections 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe670078e50 0x7fe67007b310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6a0155ef0 0x7fe6a0140b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe6a01562d0 0x7fe6a014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 --2- 10.20.193.18:0/1605784181 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6a00cde80 0x7fe6a01519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 1 librados: init done 2026-02-24T18:35:34.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 >> 10.20.193.18:0/1605784181 conn(0x7fe6a006d8c0 msgr2=0x7fe6a014b340 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 shutdown_connections 2026-02-24T18:35:34.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 -- 10.20.193.18:0/1605784181 wait complete. 2026-02-24T18:35:34.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7fe6a5832640 1 librados: shutdown 2026-02-24T18:35:34.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.412+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a38074fa0 con 0x7f5a3815fad0 2026-02-24T18:35:34.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247809 for osd.3 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.416+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9b54024060 con 0x7f9b601527f0 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.417+0000 7f9b46ffd640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 0x7f9b2c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.417+0000 7f9b5dd74640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 0x7f9b2c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.417+0000 7f9b5dd74640 10 monclient: get_auth_request con 0x7f9b2c078e50 auth_method 0 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.417+0000 7f9b46ffd640 10 client.4490.objecter ms_handle_connect 0x7f9b2c078e50 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.417+0000 7f9b5dd74640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 0x7f9b2c07b310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b4802ee20 tx=0x7f9b48033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.418+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f9b5405e020 con 0x7f9b601527f0 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.418+0000 7f9b46ffd640 10 client.4490.objecter ms_dispatch2 0x7f9b60000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.418+0000 7f9b46ffd640 3 client.4490.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.418+0000 7f9b46ffd640 3 client.4490.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.418+0000 7f9b46ffd640 20 client.4490.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.6 2026-02-24T18:35:34.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.426+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9b54061fe0 con 0x7f9b601527f0 2026-02-24T18:35:34.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.426+0000 7f9b46ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.426+0000 7f9b46ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.429+0000 7f8210ff9640 1 -- 10.20.193.18:0/1025406548 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f8204087520 con 0x7f821415d2c0 2026-02-24T18:35:34.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.429+0000 7f8210ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:34.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.429+0000 7f8210ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.431 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:34.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8004480 con 0x7f90e80c6190 2026-02-24T18:35:34.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: dump: 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d8010070 con 0x7f90e80c6190 2026-02-24T18:35:34.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8002920 con 0x7f90e80c6190 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d7fff640 4 set_mon_vals no callback set 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90d67fc640 10 monclient: dump: 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.435 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 5 monclient: authenticate success, global_id 4487 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 10 monclient: _renew_subs 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e8153de0 con 0x7f90e80c6190 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 10 client.4487.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 10 monclient: _renew_subs 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e81f7550 con 0x7f90e80c6190 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f90eff96640 1 librados: init done 2026-02-24T18:35:34.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f821a9d3640 10 librados: watch_flush enter 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.431+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800c7c0 con 0x7f1844152bb0 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: dump: 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.437 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838010070 con 0x7f1844152bb0 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800cba0 con 0x7f1844152bb0 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:34.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f182effd640 10 monclient: dump: 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 5 monclient: authenticate success, global_id 4493 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f1848dd3640 4 set_mon_vals no callback set 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _renew_subs 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1844155230 con 0x7f1844152bb0 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 10 librados: watch_flush exit 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 20 client.4469.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 10 client.4469.objecter successfully canceled tick 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 msgr2=0x7f81e407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 0x7f81e407b340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82000098a0 tx=0x7f8200006d90 comp rx=0 tx=0).stop 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 10 monclient: shutdown 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 msgr2=0x7f82141f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f820402ef10 tx=0x7f82040047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 client.4493.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _renew_subs 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441557f0 con 0x7f1844152bb0 2026-02-24T18:35:34.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 1 librados: init done 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 shutdown_connections 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f81e4078e80 0x7f81e407b340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f821415d730 0x7f82140c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f821415d2c0 0x7f82141f4540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 --2- 10.20.193.18:0/1025406548 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82141527f0 0x7f821414e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 >> 10.20.193.18:0/1025406548 conn(0x7f82140ce730 msgr2=0x7f8214151740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 shutdown_connections 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f90eff96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f90eff96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f90eff96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 -- 10.20.193.18:0/1025406548 wait complete. 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f821a9d3640 1 librados: shutdown 2026-02-24T18:35:34.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e80c9770 con 0x7f90e80c6190 2026-02-24T18:35:34.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.432+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18440c5a80 con 0x7f1844152bb0 2026-02-24T18:35:34.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.14 2026-02-24T18:35:34.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.456+0000 7f087ecf5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:34.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.456+0000 7f087ecf5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:34.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.456+0000 7f087ecf5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.456+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0878074fa0 con 0x7f0878073530 2026-02-24T18:35:34.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.477+0000 7f1dceffd640 1 -- 10.20.193.18:0/2932026534 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f1ddc0877c0 con 0x7f1de81527f0 2026-02-24T18:35:34.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.477+0000 7f1dceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:34.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.477+0000 7f1dceffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.479 INFO:teuthology.orchestra.run.trial018.stdout:98784247809 2026-02-24T18:35:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 10 librados: watch_flush enter 2026-02-24T18:35:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 10 librados: watch_flush exit 2026-02-24T18:35:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 20 client.4503.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 10 client.4503.objecter successfully canceled tick 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 msgr2=0x7f1db407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 0x7f1db407b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1de815efc0 tx=0x7f1dd0006d90 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 10 monclient: shutdown 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 msgr2=0x7f1de80c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009a70 tx=0x7f1ddc0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 shutdown_connections 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1db4078e50 0x7f1db407b310 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1de815d2c0 0x7f1de80c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1de815d730 0x7f1de81f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 --2- 10.20.193.18:0/2932026534 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1de81527f0 0x7f1de80c8b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 >> 10.20.193.18:0/2932026534 conn(0x7f1de80ce730 msgr2=0x7f1de80cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 shutdown_connections 2026-02-24T18:35:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 -- 10.20.193.18:0/2932026534 wait complete. 2026-02-24T18:35:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.479+0000 7f1deca74640 1 librados: shutdown 2026-02-24T18:35:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.487+0000 7fc896198640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.487+0000 7fc896198640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.487+0000 7fc896198640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.487+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc890155ea0 con 0x7fc890072480 2026-02-24T18:35:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.489+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5bd4002ac0 con 0x7f5be8073530 2026-02-24T18:35:34.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.489+0000 7f5bc3fff640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 0x7f5bb807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247809 for osd.15 2026-02-24T18:35:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.489+0000 7f5be6ffd640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 0x7f5bb807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.490+0000 7f5be6ffd640 10 monclient: get_auth_request con 0x7f5bb8078e00 auth_method 0 2026-02-24T18:35:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.490+0000 7f5bc3fff640 10 client.4515.objecter ms_handle_connect 0x7f5bb8078e00 2026-02-24T18:35:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.490+0000 7f5be6ffd640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 0x7f5bb807b2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5be814c0c0 tx=0x7f5bd8006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f5bd405dde0 con 0x7f5be8073530 2026-02-24T18:35:34.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 10 client.4515.objecter ms_dispatch2 0x7f5be8000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 3 client.4515.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 3 client.4515.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 20 client.4515.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5bd40624a0 con 0x7f5be8073530 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.499+0000 7f5bc3fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.505+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c03c040 con 0x7f4518152b30 2026-02-24T18:35:34.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.505+0000 7f44effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.505+0000 7f44effff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:34.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.505+0000 7f44effff640 10 monclient: dump: 2026-02-24T18:35:34.508 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: _finish_auth 0 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:34.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c004230 con 0x7f4518152b30 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c003710 con 0x7f4518152b30 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 5 monclient: authenticate success, global_id 4521 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _renew_subs 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f45157fa640 4 set_mon_vals no callback set 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4518156380 con 0x7f4518152b30 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f44effff640 10 monclient: dump: 2026-02-24T18:35:34.510 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 client.4521.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _renew_subs 2026-02-24T18:35:34.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45180c26b0 con 0x7f4518152b30 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 1 librados: init done 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.506+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451815b340 con 0x7f4518152b30 2026-02-24T18:35:34.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.513+0000 7f9b64bb3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:34.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.513+0000 7f9b64bb3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:34.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.513+0000 7f9b64bb3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.513+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f9b600c5210 con 0x7f9b601527f0 2026-02-24T18:35:34.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.514+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5a34041a10 con 0x7f5a3815fad0 2026-02-24T18:35:34.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.515+0000 7f5a25ffb640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 0x7f5a0807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.515+0000 7f5a3d26a640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 0x7f5a0807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.515+0000 7f5a3d26a640 10 monclient: get_auth_request con 0x7f5a08078e00 auth_method 0 2026-02-24T18:35:34.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.515+0000 7f5a25ffb640 10 client.4480.objecter ms_handle_connect 0x7f5a08078e00 2026-02-24T18:35:34.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.515+0000 7f5a3d26a640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 0x7f5a0807b2c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a28002c00 tx=0x7f5a28002a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.516+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f5a34083150 con 0x7f5a3815fad0 2026-02-24T18:35:34.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.516+0000 7f5a25ffb640 10 client.4480.objecter ms_dispatch2 0x7f5a38000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.516+0000 7f5a25ffb640 3 client.4480.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.516+0000 7f5a25ffb640 3 client.4480.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.516+0000 7f5a25ffb640 20 client.4480.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.522+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5a34087810 con 0x7f5a3815fad0 2026-02-24T18:35:34.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.522+0000 7f5a25ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.522+0000 7f5a25ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.524+0000 7fc86bfff640 1 -- 10.20.193.18:0/405818648 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fc87c061cb0 con 0x7fc890072480 2026-02-24T18:35:34.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.524+0000 7fc86bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:34.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.524+0000 7fc86bfff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.526 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 10 librados: watch_flush enter 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 10 librados: watch_flush exit 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 20 client.4471.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 10 client.4471.objecter successfully canceled tick 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 msgr2=0x7fc86007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 0x7fc86007b340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc880006fd0 tx=0x7fc880006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 10 monclient: shutdown 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 msgr2=0x7fc890149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00b500 tx=0x7fc87c00b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 shutdown_connections 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc860078e80 0x7fc86007b340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8901590a0 0x7fc8901f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8900739e0 0x7fc8901f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 --2- 10.20.193.18:0/405818648 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc890072480 0x7fc890149c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 >> 10.20.193.18:0/405818648 conn(0x7fc8900cd780 msgr2=0x7fc8901566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 shutdown_connections 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 -- 10.20.193.18:0/405818648 wait complete. 2026-02-24T18:35:34.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.526+0000 7fc896198640 1 librados: shutdown 2026-02-24T18:35:34.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.527+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f90d801d030 con 0x7f90e80c6190 2026-02-24T18:35:34.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.528+0000 7f90d67fc640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 0x7f90b807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.528+0000 7f90ed50a640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 0x7f90b807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.528+0000 7f90ed50a640 10 monclient: get_auth_request con 0x7f90b8078e50 auth_method 0 2026-02-24T18:35:34.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.528+0000 7f90ed50a640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 0x7f90b807b310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e81491c0 tx=0x7f90dc006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.528+0000 7f90d67fc640 10 client.4487.objecter ms_handle_connect 0x7f90b8078e50 2026-02-24T18:35:34.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.11 2026-02-24T18:35:34.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.543+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f90d801de30 con 0x7f90e80c6190 2026-02-24T18:35:34.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.543+0000 7f90d67fc640 10 client.4487.objecter ms_dispatch2 0x7f90e8000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.543+0000 7f90d67fc640 3 client.4487.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.543+0000 7f90d67fc640 3 client.4487.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.543+0000 7f90d67fc640 20 client.4487.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.584+0000 7f5bede0b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:34.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.584+0000 7f5bede0b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:34.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.584+0000 7f5bede0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.584+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5be815ddf0 con 0x7f5be8073530 2026-02-24T18:35:34.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.611+0000 7f5a3f4f5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:34.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.611+0000 7f5a3f4f5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:34.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.611+0000 7f5a3f4f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:34.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.611+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5a380c5210 con 0x7f5a3815fad0 2026-02-24T18:35:34.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.614+0000 7f5a25ffb640 1 -- 10.20.193.18:0/2876715624 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a34087110 con 0x7f5a3815fad0 2026-02-24T18:35:34.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.614+0000 7f5a25ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:34.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.614+0000 7f5a25ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.616 INFO:teuthology.orchestra.run.trial018.stdout:90194313219 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 10 librados: watch_flush enter 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 10 librados: watch_flush exit 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 20 client.4480.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 10 client.4480.objecter successfully canceled tick 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 msgr2=0x7f5a0807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 0x7f5a0807b2c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a28002c00 tx=0x7f5a28002a50 comp rx=0 tx=0).stop 2026-02-24T18:35:34.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 10 monclient: shutdown 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 msgr2=0x7f5a381f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 0x7f5a381f4130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a34036000 tx=0x7f5a34004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 shutdown_connections 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5a08078e00 0x7f5a0807b2c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5a3815fad0 0x7f5a381f4130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5a381527f0 0x7f5a38159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 --2- 10.20.193.18:0/2876715624 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5a38073530 0x7f5a38149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 >> 10.20.193.18:0/2876715624 conn(0x7f5a380ce730 msgr2=0x7f5a380cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 shutdown_connections 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 -- 10.20.193.18:0/2876715624 wait complete. 2026-02-24T18:35:34.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.616+0000 7f5a3f4f5640 1 librados: shutdown 2026-02-24T18:35:34.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.2 2026-02-24T18:35:34.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.648+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f90d8024080 con 0x7f90e80c6190 2026-02-24T18:35:34.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.648+0000 7f90d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.648+0000 7f90d67fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.656+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1838004070 con 0x7f1844152bb0 2026-02-24T18:35:34.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.656+0000 7f182effd640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 0x7f181407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.656+0000 7f184a5d6640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 0x7f181407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.656+0000 7f184a5d6640 10 monclient: get_auth_request con 0x7f1814078e50 auth_method 0 2026-02-24T18:35:34.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.657+0000 7f184a5d6640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 0x7f181407b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f183c006fd0 tx=0x7f183c006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.657+0000 7f182effd640 10 client.4493.objecter ms_handle_connect 0x7f1814078e50 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f1838012c40 con 0x7f1844152bb0 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f182effd640 10 client.4493.objecter ms_dispatch2 0x7f1844000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f182effd640 3 client.4493.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f182effd640 3 client.4493.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f182effd640 20 client.4493.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f9b46ffd640 1 -- 10.20.193.18:0/3774607672 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f9b540662e0 con 0x7f9b601527f0 2026-02-24T18:35:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f9b46ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:34.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.658+0000 7f9b46ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.661 INFO:teuthology.orchestra.run.trial018.stdout:90194313219 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 10 librados: watch_flush enter 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 10 librados: watch_flush exit 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 20 client.4490.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 10 client.4490.objecter successfully canceled tick 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 msgr2=0x7f9b2c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 0x7f9b2c07b310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b4802ee20 tx=0x7f9b48033000 comp rx=0 tx=0).stop 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 10 monclient: shutdown 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 msgr2=0x7f9b60156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 0x7f9b60156930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b54006fd0 tx=0x7f9b5400ed70 comp rx=0 tx=0).stop 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 shutdown_connections 2026-02-24T18:35:34.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9b2c078e50 0x7f9b2c07b310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9b6015d730 0x7f9b60158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9b6015d2c0 0x7f9b60157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 --2- 10.20.193.18:0/3774607672 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9b601527f0 0x7f9b60156930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 >> 10.20.193.18:0/3774607672 conn(0x7f9b600ce730 msgr2=0x7f9b600cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 shutdown_connections 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 -- 10.20.193.18:0/3774607672 wait complete. 2026-02-24T18:35:34.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.660+0000 7f9b64bb3640 1 librados: shutdown 2026-02-24T18:35:34.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.664+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f18380623a0 con 0x7f1844152bb0 2026-02-24T18:35:34.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.664+0000 7f182effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.664+0000 7f182effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.10 2026-02-24T18:35:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.711+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1f5001d030 con 0x7f1f5415f900 2026-02-24T18:35:34.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.711+0000 7f1f497fa640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 0x7f1f2407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.711+0000 7f1f58bcd640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 0x7f1f2407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.711+0000 7f1f58bcd640 10 monclient: get_auth_request con 0x7f1f24078e50 auth_method 0 2026-02-24T18:35:34.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.712+0000 7f1f497fa640 10 client.4527.objecter ms_handle_connect 0x7f1f24078e50 2026-02-24T18:35:34.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.712+0000 7f1f58bcd640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 0x7f1f2407b310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f44002c00 tx=0x7f1f44002a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f1f5001dde0 con 0x7f1f5415f900 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 10 client.4527.objecter ms_dispatch2 0x7f1f54000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 3 client.4527.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 3 client.4527.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 20 client.4527.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1f50061fc0 con 0x7f1f5415f900 2026-02-24T18:35:34.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.720+0000 7f1f497fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.735+0000 7f90eff96640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:34.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.735+0000 7f90eff96640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:34.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.735+0000 7f90eff96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.737 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.735+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f90e8147100 con 0x7f90e80c6190 2026-02-24T18:35:34.738 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.737+0000 7f90d67fc640 1 -- 10.20.193.18:0/2366333145 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f90d8061b00 con 0x7f90e80c6190 2026-02-24T18:35:34.738 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.737+0000 7f90d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:34.738 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.737+0000 7f90d67fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.739 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 10 librados: watch_flush enter 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 10 librados: watch_flush exit 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 20 client.4487.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 10 client.4487.objecter successfully canceled tick 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 msgr2=0x7f90b807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 0x7f90b807b310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e81491c0 tx=0x7f90dc006ca0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 10 monclient: shutdown 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 msgr2=0x7f90e81f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b7b0 tx=0x7f90d800bc80 comp rx=0 tx=0).stop 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.738+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 shutdown_connections 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f90b8078e50 0x7f90b807b310 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f90e81527d0 0x7f90e81531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f90e8152bb0 0x7f90e8157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 --2- 10.20.193.18:0/2366333145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f90e80c6190 0x7f90e81f3b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 >> 10.20.193.18:0/2366333145 conn(0x7f90e80cd780 msgr2=0x7f90e80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 shutdown_connections 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 -- 10.20.193.18:0/2366333145 wait complete. 2026-02-24T18:35:34.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.739+0000 7f90eff96640 1 librados: shutdown 2026-02-24T18:35:34.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.13 2026-02-24T18:35:34.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.751+0000 7f184c861640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:34.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.751+0000 7f184c861640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:34.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.751+0000 7f184c861640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:34.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.751+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f18441562d0 con 0x7f1844152bb0 2026-02-24T18:35:34.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.753+0000 7f182effd640 1 -- 10.20.193.18:0/3599788784 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1838061ca0 con 0x7f1844152bb0 2026-02-24T18:35:34.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.753+0000 7f182effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:34.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.753+0000 7f182effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.755 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.754+0000 7f184c861640 10 librados: watch_flush enter 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.754+0000 7f184c861640 10 librados: watch_flush exit 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.754+0000 7f184c861640 20 client.4493.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.754+0000 7f184c861640 10 client.4493.objecter successfully canceled tick 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 msgr2=0x7f181407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 0x7f181407b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f183c006fd0 tx=0x7f183c006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 10 monclient: shutdown 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 msgr2=0x7f18441f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f183800b9f0 tx=0x7f183800bec0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 shutdown_connections 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1814078e50 0x7f181407b310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f18441527d0 0x7f18441f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1844152bb0 0x7f18441f54b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 --2- 10.20.193.18:0/3599788784 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f18440c5fc0 0x7f1844073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 >> 10.20.193.18:0/3599788784 conn(0x7f18440cd780 msgr2=0x7f18440cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 shutdown_connections 2026-02-24T18:35:34.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 -- 10.20.193.18:0/3599788784 wait complete. 2026-02-24T18:35:34.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.755+0000 7f184c861640 1 librados: shutdown 2026-02-24T18:35:34.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.8 2026-02-24T18:35:34.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.809+0000 7f1f5ae58640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:34.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.809+0000 7f1f5ae58640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:34.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.809+0000 7f1f5ae58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.809+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1f5414cf30 con 0x7f1f5415f900 2026-02-24T18:35:34.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.831+0000 7f5bc3fff640 1 -- 10.20.193.18:0/2446358923 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5be815ddf0 con 0x7f5be8073530 2026-02-24T18:35:34.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.831+0000 7f5bc3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:34.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.831+0000 7f5bc3fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.832 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 10 librados: watch_flush enter 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 10 librados: watch_flush exit 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 20 client.4515.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 10 client.4515.objecter successfully canceled tick 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 msgr2=0x7f5bb807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 0x7f5bb807b2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5be814c0c0 tx=0x7f5bd8006cf0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 10 monclient: shutdown 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 msgr2=0x7f5be81e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b500 tx=0x7f5bd400b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 shutdown_connections 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5bb8078e00 0x7f5bb807b2c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5be81527f0 0x7f5be8145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5be815f900 0x7f5be81e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 --2- 10.20.193.18:0/2446358923 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5be8073530 0x7f5be81e7370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 >> 10.20.193.18:0/2446358923 conn(0x7f5be80ce730 msgr2=0x7f5be80cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 shutdown_connections 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 -- 10.20.193.18:0/2446358923 wait complete. 2026-02-24T18:35:34.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.833+0000 7f5bede0b640 1 librados: shutdown 2026-02-24T18:35:34.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.835+0000 7f1f497fa640 1 -- 10.20.193.18:0/2826628869 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f500618c0 con 0x7f1f5415f900 2026-02-24T18:35:34.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.835+0000 7f1f497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:34.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.835+0000 7f1f497fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:34.837 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:34.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 10 librados: watch_flush enter 2026-02-24T18:35:34.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 10 librados: watch_flush exit 2026-02-24T18:35:34.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 20 client.4527.objecter shutdown clearing up homeless session... 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 10 client.4527.objecter successfully canceled tick 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 msgr2=0x7f1f2407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 0x7f1f2407b310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f44002c00 tx=0x7f1f44002a50 comp rx=0 tx=0).stop 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 10 monclient: shutdown 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 msgr2=0x7f1f54157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.837+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b990 tx=0x7f1f5000be60 comp rx=0 tx=0).stop 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 shutdown_connections 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1f24078e50 0x7f1f2407b310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1f5415f900 0x7f1f54157e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1f541527f0 0x7f1f541578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 --2- 10.20.193.18:0/2826628869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1f54073530 0x7f1f540c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:34.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 >> 10.20.193.18:0/2826628869 conn(0x7f1f540ce730 msgr2=0x7f1f540cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:34.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 shutdown_connections 2026-02-24T18:35:34.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 -- 10.20.193.18:0/2826628869 wait complete. 2026-02-24T18:35:34.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.838+0000 7f1f5ae58640 1 librados: shutdown 2026-02-24T18:35:34.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.4 2026-02-24T18:35:34.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.7 2026-02-24T18:35:34.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.847+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efc00007030 con 0x7efc041f5730 2026-02-24T18:35:34.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.848+0000 7efbf97fa640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 0x7efbd407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.848+0000 7efc08f51640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 0x7efbd407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.848+0000 7efc08f51640 10 monclient: get_auth_request con 0x7efbd4078e50 auth_method 0 2026-02-24T18:35:34.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.848+0000 7efbf97fa640 10 client.4524.objecter ms_handle_connect 0x7efbd4078e50 2026-02-24T18:35:34.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.848+0000 7efc08f51640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 0x7efbd407b310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efbf4007ab0 tx=0x7efbf4002a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.854+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2b14007490 con 0x7f2b200ceca0 2026-02-24T18:35:34.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.855+0000 7f2b0dffb640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 0x7f2af007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.855+0000 7f2b25399640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 0x7f2af007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.855+0000 7f2b25399640 10 monclient: get_auth_request con 0x7f2af0078e50 auth_method 0 2026-02-24T18:35:34.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.856+0000 7f2b0dffb640 10 client.4518.objecter ms_handle_connect 0x7f2af0078e50 2026-02-24T18:35:34.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.856+0000 7f2b25399640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 0x7f2af007b310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b10039000 tx=0x7f2b10033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.870+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f2b140077b0 con 0x7f2b200ceca0 2026-02-24T18:35:34.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.870+0000 7f2b0dffb640 10 client.4518.objecter ms_dispatch2 0x7f2b20000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.870+0000 7f2b0dffb640 3 client.4518.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.870+0000 7f2b0dffb640 3 client.4518.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.871+0000 7f2b0dffb640 20 client.4518.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.871+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2b14021080 con 0x7f2b200ceca0 2026-02-24T18:35:34.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.871+0000 7f2b0dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.871+0000 7f2b0dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7efc0005dc10 con 0x7efc041f5730 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 10 client.4524.objecter ms_dispatch2 0x7efc04000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 3 client.4524.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 3 client.4524.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 20 client.4524.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efc0001a060 con 0x7efc041f5730 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.877+0000 7efbf97fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:34.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.955+0000 7f2b27624640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:34.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.955+0000 7f2b27624640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:34.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.955+0000 7f2b27624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.955+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f2b20159810 con 0x7f2b200ceca0 2026-02-24T18:35:34.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.963+0000 7efc0b1dc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:34.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.963+0000 7efc0b1dc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:34.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.963+0000 7efc0b1dc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:34.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.963+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7efc0415ee90 con 0x7efc041f5730 2026-02-24T18:35:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.987+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f450c03f050 con 0x7f4518152b30 2026-02-24T18:35:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.987+0000 7f44effff640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 0x7f44e807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:34.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.988+0000 7f45167fc640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 0x7f44e807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:34.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.988+0000 7f45167fc640 10 monclient: get_auth_request con 0x7f44e8078e50 auth_method 0 2026-02-24T18:35:34.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.988+0000 7f44effff640 10 client.4521.objecter ms_handle_connect 0x7f44e8078e50 2026-02-24T18:35:34.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.988+0000 7f45167fc640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 0x7f44e807b310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4500006fd0 tx=0x7f4500006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:34.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f450c02fbb0 con 0x7f4518152b30 2026-02-24T18:35:34.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 10 client.4521.objecter ms_dispatch2 0x7f4518000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:34.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 3 client.4521.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:34.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 3 client.4521.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:34.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 20 client.4521.objecter dump_active .. 0 homeless 2026-02-24T18:35:34.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f450c044020 con 0x7f4518152b30 2026-02-24T18:35:34.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:34.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f44effff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f086d7fa640 1 -- 10.20.193.18:0/3054785757 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08780739a0 con 0x7f0878073530 2026-02-24T18:35:35.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f086d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:35.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.996+0000 7f086d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.000 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:35.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 10 librados: watch_flush enter 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 10 librados: watch_flush exit 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 20 client.4506.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 10 client.4506.objecter successfully canceled tick 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 msgr2=0x7f084807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 0x7f084807b340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0860007fd0 tx=0x7f0860007b40 comp rx=0 tx=0).stop 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 10 monclient: shutdown 2026-02-24T18:35:35.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 msgr2=0x7f08781f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f086800c700 tx=0x7f086800cbd0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 shutdown_connections 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0848078e80 0x7f084807b340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f08781527f0 0x7f08780c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f087815fad0 0x7f08781f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 --2- 10.20.193.18:0/3054785757 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0878073530 0x7f08781f76a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 >> 10.20.193.18:0/3054785757 conn(0x7f08780ce730 msgr2=0x7f08780ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 shutdown_connections 2026-02-24T18:35:35.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 -- 10.20.193.18:0/3054785757 wait complete. 2026-02-24T18:35:35.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:34.998+0000 7f087ecf5640 1 librados: shutdown 2026-02-24T18:35:35.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.001+0000 7f2b0dffb640 1 -- 10.20.193.18:0/1451771671 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f2b140612a0 con 0x7f2b200ceca0 2026-02-24T18:35:35.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.001+0000 7f2b0dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:35.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.001+0000 7f2b0dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.006 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 10 librados: watch_flush enter 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 10 librados: watch_flush exit 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 20 client.4518.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 10 client.4518.objecter successfully canceled tick 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 msgr2=0x7f2af007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 0x7f2af007b310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b10039000 tx=0x7f2b10033000 comp rx=0 tx=0).stop 2026-02-24T18:35:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 10 monclient: shutdown 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 msgr2=0x7f2b201f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 0x7f2b201f4590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400c460 tx=0x7f2b1400c930 comp rx=0 tx=0).stop 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 shutdown_connections 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2af0078e50 0x7f2af007b310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2b2015f650 0x7f2b201f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2b200ceca0 0x7f2b201f4590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 --2- 10.20.193.18:0/1451771671 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2b200c9b00 0x7f2b200d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 >> 10.20.193.18:0/1451771671 conn(0x7f2b20072fc0 msgr2=0x7f2b200cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 shutdown_connections 2026-02-24T18:35:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 -- 10.20.193.18:0/1451771671 wait complete. 2026-02-24T18:35:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.003+0000 7f2b27624640 1 librados: shutdown 2026-02-24T18:35:35.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.1 2026-02-24T18:35:35.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247811 for osd.12 2026-02-24T18:35:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.031+0000 7efbf97fa640 1 -- 10.20.193.18:0/805153488 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc000102c0 con 0x7efc041f5730 2026-02-24T18:35:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.031+0000 7efbf97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.031+0000 7efbf97fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.033 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 10 librados: watch_flush enter 2026-02-24T18:35:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 10 librados: watch_flush exit 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 20 client.4524.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 10 client.4524.objecter successfully canceled tick 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 msgr2=0x7efbd407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 0x7efbd407b310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efbf4007ab0 tx=0x7efbf4002a60 comp rx=0 tx=0).stop 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 10 monclient: shutdown 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 msgr2=0x7efc04148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc040ce270 tx=0x7efc0000cf10 comp rx=0 tx=0).stop 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 shutdown_connections 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd4078e50 0x7efbd407b310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc040ca390 0x7efc041f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc041f5730 0x7efc04148fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 --2- 10.20.193.18:0/805153488 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efbf0004900 0x7efc040c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 >> 10.20.193.18:0/805153488 conn(0x7efc040ce730 msgr2=0x7efc040cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 shutdown_connections 2026-02-24T18:35:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 -- 10.20.193.18:0/805153488 wait complete. 2026-02-24T18:35:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.033+0000 7efc0b1dc640 1 librados: shutdown 2026-02-24T18:35:35.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.9 2026-02-24T18:35:35.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.086+0000 7f451d911640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:35.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.086+0000 7f451d911640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:35.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.086+0000 7f451d911640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.086+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4518068aa0 con 0x7f4518152b30 2026-02-24T18:35:35.307 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:35:35.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 10 monclient: monmap: 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.379170+0000 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.379170+0000 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.380 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.381 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.381 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.381 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 1 Processor -- start 2026-02-24T18:35:35.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 1 -- start start 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.378+0000 7f6a5d0d3640 10 monclient: init 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient: _add_conns ranks=[0,4,3,2,1,5] 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6a5806ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6a580c99a0 0x7f6a58070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-b con 0x7f6a580c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6a5815a5e0 0x7f6a5814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-a con 0x7f6a5815a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580c7e90 con 0x7f6a5806ff20 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6a58159a40 con 0x7f6a5815a5e0 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6a580c8170 con 0x7f6a580c99a0 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a5d0d3640 10 monclient(hunting): authenticate will time out at 1479.060326s 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.379+0000 7f6a56575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6a580c99a0 0x7f6a58070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6a580c99a0 0x7f6a58070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53386/0 (socket says 10.20.193.18:53386) 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 1 -- 10.20.193.18:0/4169749428 learned_addr learned my addr 10.20.193.18:0/4169749428 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): get_auth_request con 0x7f6a580c99a0 auth_method 0 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a57577640 1 --2- 10.20.193.18:0/4169749428 >> v2:10.20.193.18:3300/0 conn(0x7f6a5815a5e0 0x7f6a5814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a57577640 10 monclient(hunting): get_auth_request con 0x7f6a5815a5e0 auth_method 0 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a57577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a57577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.380+0000 7f6a57577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 10 monclient(hunting): handle_auth_done global_id 4483 payload 274 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 1 -- 10.20.193.18:0/4169749428 >> v2:10.20.193.18:3300/0 conn(0x7f6a5815a5e0 msgr2=0x7f6a5814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 1 --2- 10.20.193.18:0/4169749428 >> v2:10.20.193.18:3300/0 conn(0x7f6a5815a5e0 0x7f6a5814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 1 -- 10.20.193.18:0/4169749428 >> v1:10.20.193.18:6789/0 conn(0x7f6a5806ff20 legacy=0x7f6a58159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.381+0000 7f6a56575640 1 -- 10.20.193.18:0/4169749428 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580c8450 con 0x7f6a580c99a0 2026-02-24T18:35:35.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.382+0000 7f6a56575640 1 --2- 10.20.193.18:0/4169749428 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a58070300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009970 tx=0x7f6a4002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd86c5f14043a280 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 1 -- 10.20.193.18:0/4169749428 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4003c070 con 0x7f6a580c99a0 2026-02-24T18:35:35.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: dump: 2026-02-24T18:35:35.392 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 5 monclient: authenticate success, global_id 4483 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 1 -- 10.20.193.18:0/4169749428 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4002fb20 con 0x7f6a580c99a0 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 1 -- 10.20.193.18:0/4169749428 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40037440 con 0x7f6a580c99a0 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 4 set_mon_vals no callback set 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a55d74640 10 monclient: dump: 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.394 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 10 monclient: shutdown 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4169749428 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 msgr2=0x7f6a58070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4169749428 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a58070300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009970 tx=0x7f6a4002eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4169749428 shutdown_connections 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4169749428 >> v2:10.20.193.18:3300/0 conn(0x7f6a5815a5e0 0x7f6a5814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4169749428 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a58070300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4169749428 >> 10.20.193.18:0/4169749428 conn(0x7f6a580cd780 msgr2=0x7f6a580cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4169749428 shutdown_connections 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4169749428 wait complete. 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 10 monclient: monmap: 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 librados: starting msgr at 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 librados: starting objecter 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.384+0000 7f6a5d0d3640 1 Processor -- start 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 -- start start 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 librados: setting wanted keys 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 librados: calling monclient init 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient: init 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a5806ff20 0x7f6a580c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-c con 0x7f6a5806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-b con 0x7f6a580c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a5815a5e0 0x7f6a581f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): picked mon.noname-a con 0x7f6a5815a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58150350 con 0x7f6a5815a5e0 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580c8a10 con 0x7f6a580c99a0 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581504d0 con 0x7f6a5806ff20 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a5d0d3640 10 monclient(hunting): authenticate will time out at 1479.066448s 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56d76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a5806ff20 0x7f6a580c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53394/0 (socket says 10.20.193.18:53394) 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 1 -- 10.20.193.18:0/4282241778 learned_addr learned my addr 10.20.193.18:0/4282241778 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 10 monclient(hunting): get_auth_request con 0x7f6a580c99a0 auth_method 0 2026-02-24T18:35:35.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a56575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.385+0000 7f6a57577640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a5815a5e0 0x7f6a581f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): get_auth_request con 0x7f6a5815a5e0 auth_method 0 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): get_auth_request con 0x7f6a5806ff20 auth_method 0 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a57577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.386+0000 7f6a56d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 10 monclient(hunting): handle_auth_done global_id 4486 payload 824 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 -- 10.20.193.18:0/4282241778 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a5806ff20 msgr2=0x7f6a580c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a5806ff20 0x7f6a580c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 -- 10.20.193.18:0/4282241778 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a5815a5e0 msgr2=0x7f6a581f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a5815a5e0 0x7f6a581f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.387+0000 7f6a56575640 1 -- 10.20.193.18:0/4282241778 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a581f4e00 con 0x7f6a580c99a0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.388+0000 7f6a56575640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009aa0 tx=0x7f6a400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.388+0000 7f6a377fe640 10 client.?.objecter ms_handle_connect 0x7f6a580c99a0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.388+0000 7f6a377fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4003c040 con 0x7f6a580c99a0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: dump: 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.401 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40042630 con 0x7f6a580c99a0 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40042a20 con 0x7f6a580c99a0 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a55573640 4 set_mon_vals no callback set 2026-02-24T18:35:35.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a377fe640 10 monclient: dump: 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 5 monclient: authenticate success, global_id 4486 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _renew_subs 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581f50e0 con 0x7f6a580c99a0 2026-02-24T18:35:35.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 client.4486.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _renew_subs 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581f56a0 con 0x7f6a580c99a0 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 1 librados: init done 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.390+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a58150860 con 0x7f6a580c99a0 2026-02-24T18:35:35.420 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-24T18:35:35.422 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-24T18:35:35.445 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.490+0000 7f2d203a9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.490+0000 7f2d203a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.490+0000 7f2d203a9640 10 monclient: monmap: 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.491505+0000 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.491505+0000 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.492 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.493 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.493 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-24T18:35:35.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 Processor -- start 2026-02-24T18:35:35.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 -- start start 2026-02-24T18:35:35.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient: init 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 10 monclient: monmap: 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.492157+0000 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.492157+0000 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.495 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 1 Processor -- start 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 1 -- start start 2026-02-24T18:35:35.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f5df7b53640 10 monclient: init 2026-02-24T18:35:35.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient: _add_conns ranks=[1,0,3,5,2,4] 2026-02-24T18:35:35.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2d181527d0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d18152bb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2d180c6190 0x7f2d180c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-a con 0x7f2d180c6190 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1806e300 con 0x7f2d18152bb0 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1806e5e0 con 0x7f2d181527d0 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1806f1f0 con 0x7f2d180c6190 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.491+0000 7f2d203a9640 10 monclient(hunting): authenticate will time out at 1479.172981s 2026-02-24T18:35:35.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2d180c6190 0x7f2d180c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient: _add_conns ranks=[4,1,2,5,0,3] 2026-02-24T18:35:35.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f5df006ff20 0x7f5df0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-b con 0x7f5df006ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5df00c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5df015a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df00c7e90 con 0x7f5df00c99a0 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df00c8170 con 0x7f5df015a5e0 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5df0159a40 con 0x7f5df006ff20 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f5df7b53640 10 monclient(hunting): authenticate will time out at 1479.173686s 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2d180c6190 0x7f2d180c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41604/0 (socket says 10.20.193.18:41604) 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 1 -- 10.20.193.18:0/3672964336 learned_addr learned my addr 10.20.193.18:0/3672964336 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 10 monclient(hunting): get_auth_request con 0x7f2d180c6190 auth_method 0 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.492+0000 7f2d1e91f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f5df60c9640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f5df015a5e0 0x7f5df014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58350/0 (socket says 10.20.193.18:58350) 2026-02-24T18:35:35.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f5df60c9640 1 -- 10.20.193.18:0/2932363569 learned_addr learned my addr 10.20.193.18:0/2932363569 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.493+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6a40042bc0 con 0x7f6a580c99a0 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f6a377fe640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 0x7f6a2407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f6a56d76640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 0x7f6a2407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f6a56d76640 10 monclient: get_auth_request con 0x7f6a24078e50 auth_method 0 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f6a377fe640 10 client.4486.objecter ms_handle_connect 0x7f6a24078e50 2026-02-24T18:35:35.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f6a56d76640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 0x7f6a2407b310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0046e0 tx=0x7f6a4c0091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_done global_id 4542 payload 274 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 1 -- 10.20.193.18:0/3672964336 >> v1:10.20.193.32:6789/0 conn(0x7f2d181527d0 legacy=0x7f2d1806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 1 -- 10.20.193.18:0/3672964336 >> v1:10.20.193.18:6789/0 conn(0x7f2d18152bb0 legacy=0x7f2d180c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.494+0000 7f2d1e91f640 1 -- 10.20.193.18:0/3672964336 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1806e8c0 con 0x7f2d180c6190 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.495+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f6a4002f9e0 con 0x7f6a580c99a0 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.495+0000 7f6a377fe640 10 client.4486.objecter ms_dispatch2 0x7f6a58000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.495+0000 7f6a377fe640 3 client.4486.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.495+0000 7f6a377fe640 3 client.4486.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.495+0000 7f6a377fe640 20 client.4486.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.496+0000 7f2d1e91f640 1 --2- 10.20.193.18:0/3672964336 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180c8e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d1806f370 tx=0x7f2d14031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2971fa19cdee05ae server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.501+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6a40087750 con 0x7f6a580c99a0 2026-02-24T18:35:35.507 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.501+0000 7f6a377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.507 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.501+0000 7f6a377fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.502+0000 7f5df58c8640 1 --2- 10.20.193.18:0/2932363569 >> v2:10.20.193.32:3300/0 conn(0x7f5df006ff20 0x7f5df0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): get_auth_request con 0x7f5df006ff20 auth_method 0 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.503+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 10 monclient(hunting): handle_auth_done global_id 4489 payload 274 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 1 -- 10.20.193.18:0/2932363569 >> v1:10.20.193.110:6789/0 conn(0x7f5df015a5e0 legacy=0x7f5df014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 1 -- 10.20.193.18:0/2932363569 >> v1:10.20.193.32:6789/0 conn(0x7f5df00c99a0 legacy=0x7f5df0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.504+0000 7f5df58c8640 1 -- 10.20.193.18:0/2932363569 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df00c8450 con 0x7f5df006ff20 2026-02-24T18:35:35.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.506+0000 7f5df58c8640 1 --2- 10.20.193.18:0/2932363569 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df006ff20 0x7f5df0070300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f038c3acfc8ddf8d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 1 -- 10.20.193.18:0/2932363569 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de003c070 con 0x7f5df006ff20 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: dump: 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.511 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 1 -- 10.20.193.18:0/2932363569 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de002fba0 con 0x7f5df006ff20 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 1 -- 10.20.193.18:0/2932363569 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0037440 con 0x7f5df006ff20 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df48c6640 10 monclient: dump: 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.512 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 5 monclient: authenticate success, global_id 4489 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 4 set_mon_vals no callback set 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 10 monclient: shutdown 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 -- 10.20.193.18:0/2932363569 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df006ff20 msgr2=0x7f5df0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2932363569 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df006ff20 0x7f5df0070300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de002eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 -- 10.20.193.18:0/2932363569 shutdown_connections 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2932363569 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df006ff20 0x7f5df0070300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 -- 10.20.193.18:0/2932363569 >> 10.20.193.18:0/2932363569 conn(0x7f5df00cd780 msgr2=0x7f5df00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 -- 10.20.193.18:0/2932363569 shutdown_connections 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.508+0000 7f5df7b53640 1 -- 10.20.193.18:0/2932363569 wait complete. 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient: monmap: 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.514 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 librados: starting msgr at 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 librados: starting objecter 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 Processor -- start 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 -- start start 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 librados: setting wanted keys 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 librados: calling monclient init 2026-02-24T18:35:35.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient: init 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5df006ff20 0x7f5df01f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-c con 0x7f5df006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5df00c99a0 0x7f5df00c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-a con 0x7f5df00c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): picked mon.noname-b con 0x7f5df015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0150520 con 0x7f5df00c99a0 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00c8a10 con 0x7f5df015a5e0 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.509+0000 7f5df7b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df7b53640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01506a0 con 0x7f5df006ff20 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df7b53640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df7b53640 10 monclient(hunting): authenticate will time out at 1479.191016s 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5df00c99a0 0x7f5df00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53410/0 (socket says 10.20.193.18:53410) 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 1 -- 10.20.193.18:0/2748690341 learned_addr learned my addr 10.20.193.18:0/2748690341 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): get_auth_request con 0x7f5df015a5e0 auth_method 0 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5df006ff20 0x7f5df01f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): get_auth_request con 0x7f5df00c99a0 auth_method 0 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df60c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): get_auth_request con 0x7f5df006ff20 auth_method 0 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df50c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.510+0000 7f5df58c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 10 monclient(hunting): handle_auth_done global_id 4492 payload 824 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 -- 10.20.193.18:0/2748690341 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5df006ff20 msgr2=0x7f5df01f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5df006ff20 0x7f5df01f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 -- 10.20.193.18:0/2748690341 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5df00c99a0 msgr2=0x7f5df00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5df00c99a0 0x7f5df00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.512+0000 7f5df60c9640 1 -- 10.20.193.18:0/2748690341 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df006f340 con 0x7f5df015a5e0 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.513+0000 7f5df60c9640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00cb10 tx=0x7f5dec00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.513+0000 7f5dde7fc640 10 client.?.objecter ms_handle_connect 0x7f5df015a5e0 2026-02-24T18:35:35.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.513+0000 7f5dde7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.514+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec017070 con 0x7f5df015a5e0 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.514+0000 7f5dde7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.514+0000 7f5dde7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.514+0000 7f5dde7fc640 10 monclient: dump: 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.523 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec00ac50 con 0x7f5df015a5e0 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec012400 con 0x7f5df015a5e0 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5dde7fc640 10 monclient: dump: 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.524 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5ddffff640 4 set_mon_vals no callback set 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 5 monclient: authenticate success, global_id 4492 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _renew_subs 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df01f4ce0 con 0x7f5df015a5e0 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 client.4492.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _renew_subs 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01f4a00 con 0x7f5df015a5e0 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 1 librados: init done 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.515+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df0070300 con 0x7f5df015a5e0 2026-02-24T18:35:35.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.525+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5dec024080 con 0x7f5df015a5e0 2026-02-24T18:35:35.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.526+0000 7f5dde7fc640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 0x7f5dc007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.526+0000 7f5df58c8640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 0x7f5dc007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.526+0000 7f5df58c8640 10 monclient: get_auth_request con 0x7f5dc0078e00 auth_method 0 2026-02-24T18:35:35.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.526+0000 7f5df58c8640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 0x7f5dc007b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5de0039000 tx=0x7f5de0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.526+0000 7f5dde7fc640 10 client.4492.objecter ms_handle_connect 0x7f5dc0078e00 2026-02-24T18:35:35.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.527+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f5dec05dc50 con 0x7f5df015a5e0 2026-02-24T18:35:35.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.527+0000 7f5dde7fc640 10 client.4492.objecter ms_dispatch2 0x7f5df0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.527+0000 7f5dde7fc640 3 client.4492.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.527+0000 7f5dde7fc640 3 client.4492.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.527+0000 7f5dde7fc640 20 client.4492.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.532+0000 7fe5a4c98640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.532+0000 7fe5a4c98640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.532+0000 7fe5a4c98640 10 monclient: monmap: 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.533601+0000 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.533601+0000 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.534 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.535 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.535 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.535 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.535 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.535 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 Processor -- start 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 -- start start 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient: init 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient: _add_conns ranks=[4,3,2,5,0,1] 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe5a0155f80 0x7fe5a015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-b con 0x7fe5a0155f80 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe5a00ccb40 0x7fe5a015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-a con 0x7fe5a00ccb40 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe5a0156360 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a00d0380 con 0x7fe5a0156360 2026-02-24T18:35:35.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a006d5b0 con 0x7fe5a00ccb40 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a006c9a0 con 0x7fe5a0155f80 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.533+0000 7fe5a4c98640 10 monclient(hunting): authenticate will time out at 1479.214823s 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe5a00ccb40 0x7fe5a015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59ed76640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fe5a0156360 0x7fe5a015f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58352/0 (socket says 10.20.193.18:58352) 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59ed76640 1 -- 10.20.193.18:0/3067304072 learned_addr learned my addr 10.20.193.18:0/3067304072 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): get_auth_request con 0x7fe5a00ccb40 auth_method 0 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.534+0000 7fe59dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.539 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 10 monclient(hunting): handle_auth_done global_id 4548 payload 274 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 1 -- 10.20.193.18:0/3067304072 >> v2:10.20.193.32:3300/0 conn(0x7fe5a0155f80 msgr2=0x7fe5a015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 1 --2- 10.20.193.18:0/3067304072 >> v2:10.20.193.32:3300/0 conn(0x7fe5a0155f80 0x7fe5a015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 1 -- 10.20.193.18:0/3067304072 >> v1:10.20.193.110:6789/0 conn(0x7fe5a0156360 legacy=0x7fe5a015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.536+0000 7fe59dd74640 1 -- 10.20.193.18:0/3067304072 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a006cc80 con 0x7fe5a00ccb40 2026-02-24T18:35:35.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.537+0000 7fe59dd74640 1 --2- 10.20.193.18:0/3067304072 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a015d320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5a006f270 tx=0x7fe58802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd42b82a59bc0e4 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.566+0000 7f15eaaea640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.566+0000 7f15eaaea640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.566+0000 7f15eaaea640 10 monclient: monmap: 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.567323+0000 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.567323+0000 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.568 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.569 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.569 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.569 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.566+0000 7f15eaaea640 1 Processor -- start 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 1 -- start start 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient: init 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient: _add_conns ranks=[2,1,4,5,3,0] 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f15e41527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:35.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f15e4073530 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f15e415fad0 0x7f15e40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-b con 0x7f15e415fad0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e40739a0 con 0x7f15e4073530 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e415d9d0 con 0x7f15e41527f0 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f15e4074b10 con 0x7f15e415fad0 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15eaaea640 10 monclient(hunting): authenticate will time out at 1479.248538s 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15e885f640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f15e41527f0 0x7f15e415d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58356/0 (socket says 10.20.193.18:58356) 2026-02-24T18:35:35.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.567+0000 7f15e885f640 1 -- 10.20.193.18:0/2328711896 learned_addr learned my addr 10.20.193.18:0/2328711896 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.597+0000 7f6a5d0d3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:35.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.597+0000 7f6a5d0d3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:35.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.597+0000 7f6a5d0d3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.597+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6a5815a070 con 0x7f6a580c99a0 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.604+0000 7efc10457640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.604+0000 7efc10457640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.604+0000 7efc10457640 10 monclient: monmap: 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.605547+0000 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.605547+0000 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.606 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.607 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.607 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.607 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.607 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.607 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 Processor -- start 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 -- start start 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient: init 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient: _add_conns ranks=[1,3,4,0,5,2] 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efc0806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efc080c99a0 0x7efc08070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): picked mon.noname-a con 0x7efc080c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efc0815a5e0 0x7efc0814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): picked mon.noname-b con 0x7efc0815a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc080c7e90 con 0x7efc0806ff20 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7efc08159a40 con 0x7efc080c99a0 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7efc080c8170 con 0x7efc0815a5e0 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.605+0000 7efc10457640 10 monclient(hunting): authenticate will time out at 1479.286711s 2026-02-24T18:35:35.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.617+0000 7efc0d9cb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efc080c99a0 0x7efc08070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efc080c99a0 0x7efc08070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41638/0 (socket says 10.20.193.18:41638) 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 1 -- 10.20.193.18:0/179638474 learned_addr learned my addr 10.20.193.18:0/179638474 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 10 monclient(hunting): get_auth_request con 0x7efc080c99a0 auth_method 0 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.620+0000 7efc0d9cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 1 -- 10.20.193.18:0/3672964336 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1403d070 con 0x7f2d180c6190 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: dump: 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.624 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 1 -- 10.20.193.18:0/3672964336 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1400b4e0 con 0x7f2d180c6190 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 1 -- 10.20.193.18:0/3672964336 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140098a0 con 0x7f2d180c6190 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d1d11c640 10 monclient: dump: 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.625 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 5 monclient: authenticate success, global_id 4542 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 4 set_mon_vals no callback set 2026-02-24T18:35:35.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7efc0d9cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7efc0d9cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7efc0d9cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 10 monclient: shutdown 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 1 -- 10.20.193.18:0/3672964336 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 msgr2=0x7f2d180c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 1 --2- 10.20.193.18:0/3672964336 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180c8e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d1806f370 tx=0x7f2d14031520 comp rx=0 tx=0).stop 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.622+0000 7f2d203a9640 1 -- 10.20.193.18:0/3672964336 shutdown_connections 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 --2- 10.20.193.18:0/3672964336 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180c8e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 -- 10.20.193.18:0/3672964336 >> 10.20.193.18:0/3672964336 conn(0x7f2d180cd780 msgr2=0x7f2d180cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 -- 10.20.193.18:0/3672964336 shutdown_connections 2026-02-24T18:35:35.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 -- 10.20.193.18:0/3672964336 wait complete. 2026-02-24T18:35:35.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 1 -- 10.20.193.18:0/3067304072 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58803c070 con 0x7fe5a00ccb40 2026-02-24T18:35:35.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: dump: 2026-02-24T18:35:35.629 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 1 -- 10.20.193.18:0/3067304072 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58802fa60 con 0x7fe5a00ccb40 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 1 -- 10.20.193.18:0/3067304072 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58802fe30 con 0x7fe5a00ccb40 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 5 monclient: authenticate success, global_id 4548 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 4 set_mon_vals no callback set 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe59d573640 10 monclient: dump: 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.631 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 10 monclient: monmap: 2026-02-24T18:35:35.632 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 librados: starting msgr at 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 librados: starting objecter 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 Processor -- start 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 -- start start 2026-02-24T18:35:35.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 librados: setting wanted keys 2026-02-24T18:35:35.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 1 librados: calling monclient init 2026-02-24T18:35:35.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7f2d203a9640 10 monclient: init 2026-02-24T18:35:35.634 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:35:35.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 10 monclient: shutdown 2026-02-24T18:35:35.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 1 -- 10.20.193.18:0/3067304072 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 msgr2=0x7fe5a015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.623+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/3067304072 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a015d320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5a006f270 tx=0x7fe58802ed50 comp rx=0 tx=0).stop 2026-02-24T18:35:35.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 -- 10.20.193.18:0/3067304072 shutdown_connections 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/3067304072 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a015d320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/3067304072 >> v2:10.20.193.32:3300/0 conn(0x7fe5a0155f80 0x7fe5a015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 -- 10.20.193.18:0/3067304072 >> 10.20.193.18:0/3067304072 conn(0x7fe5a00cc020 msgr2=0x7fe5a00cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 -- 10.20.193.18:0/3067304072 shutdown_connections 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 -- 10.20.193.18:0/3067304072 wait complete. 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 10 monclient(hunting): handle_auth_done global_id 4551 payload 274 2026-02-24T18:35:35.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 1 -- 10.20.193.18:0/179638474 >> v2:10.20.193.32:3300/0 conn(0x7efc0815a5e0 msgr2=0x7efc0814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 1 --2- 10.20.193.18:0/179638474 >> v2:10.20.193.32:3300/0 conn(0x7efc0815a5e0 0x7efc0814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 1 -- 10.20.193.18:0/179638474 >> v1:10.20.193.32:6789/0 conn(0x7efc0806ff20 legacy=0x7efc08159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7efc0d9cb640 1 -- 10.20.193.18:0/179638474 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080c8450 con 0x7efc080c99a0 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-a con 0x7f2d180c6190 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d181527d0 0x7f2d181f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-b con 0x7f2d181527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): picked mon.noname-c con 0x7f2d18152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180c93a0 con 0x7f2d180c6190 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1806ee80 con 0x7f2d181527d0 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180c9520 con 0x7f2d18152bb0 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d203a9640 10 monclient(hunting): authenticate will time out at 1479.305472s 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 10 monclient: monmap: 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.640 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 librados: starting msgr at 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 librados: starting objecter 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 Processor -- start 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 -- start start 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 librados: setting wanted keys 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 1 librados: calling monclient init 2026-02-24T18:35:35.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7fe5a4c98640 10 monclient: init 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.624+0000 7f2d1e91f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44448/0 (socket says 10.20.193.18:44448) 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 1 -- 10.20.193.18:0/1007822029 learned_addr learned my addr 10.20.193.18:0/1007822029 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): get_auth_request con 0x7f2d18152bb0 auth_method 0 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-a con 0x7fe5a00ccb40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5a0155f80 0x7fe5a00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-b con 0x7fe5a0155f80 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): picked mon.noname-c con 0x7fe5a0156360 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0072eb0 con 0x7fe5a00ccb40 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a006d240 con 0x7fe5a0155f80 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a015d860 con 0x7fe5a0156360 2026-02-24T18:35:35.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7fe5a4c98640 10 monclient(hunting): authenticate will time out at 1479.306292s 2026-02-24T18:35:35.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.625+0000 7efc0d9cb640 1 --2- 10.20.193.18:0/179638474 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc080c99a0 0x7efc08070300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009970 tx=0x7efbfc02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cf8a2acbb35113 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 10 monclient(hunting): handle_auth_done global_id 4502 payload 824 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 -- 10.20.193.18:0/1007822029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d181527d0 msgr2=0x7f2d181f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d181527d0 0x7f2d181f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 -- 10.20.193.18:0/1007822029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 msgr2=0x7f2d180cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 monclient: found mon.noname-c 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.626+0000 7f2d1e91f640 1 -- 10.20.193.18:0/1007822029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d180cb500 con 0x7f2d18152bb0 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5dec061c10 con 0x7f5df015a5e0 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7f5dde7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7f5dde7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44462/0 (socket says 10.20.193.18:44462) 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 1 -- 10.20.193.18:0/605673827 learned_addr learned my addr 10.20.193.18:0/605673827 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 10 monclient(hunting): get_auth_request con 0x7fe5a0156360 auth_method 0 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.627+0000 7fe59ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7f2d1e91f640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d250 tx=0x7f2d14004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7f2cfeffd640 10 client.?.objecter ms_handle_connect 0x7f2d18152bb0 2026-02-24T18:35:35.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7f2cfeffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7fe59ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7fe59ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.628+0000 7fe59ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.629+0000 7fe59ed76640 10 monclient(hunting): handle_auth_done global_id 4505 payload 824 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5a0155f80 msgr2=0x7fe5a00c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5a0155f80 0x7fe5a00c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 msgr2=0x7fe5a01567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 monclient: found mon.noname-c 2026-02-24T18:35:35.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7fe59ed76640 1 -- 10.20.193.18:0/605673827 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a014e1c0 con 0x7fe5a0156360 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7f6a377fe640 1 -- 10.20.193.18:0/4282241778 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a4008ba50 con 0x7f6a580c99a0 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7f6a377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.630+0000 7f6a377fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.651 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.631+0000 7f15e9060640 1 --2- 10.20.193.18:0/2328711896 >> v2:10.20.193.32:3300/0 conn(0x7f15e415fad0 0x7f15e40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.631+0000 7f15e9060640 10 monclient(hunting): get_auth_request con 0x7f15e415fad0 auth_method 0 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.631+0000 7f15e9060640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.631+0000 7f15e9060640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.631+0000 7f15e9060640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f15e9060640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f15e9060640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f15e9060640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7fe59ed76640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ca80 tx=0x7fe59000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7fe586ffd640 10 client.?.objecter ms_handle_connect 0x7fe5a0156360 2026-02-24T18:35:35.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7fe586ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 10 librados: watch_flush enter 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 10 librados: watch_flush exit 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 20 client.4486.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 10 client.4486.objecter successfully canceled tick 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 msgr2=0x7f6a2407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 0x7f6a2407b310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0046e0 tx=0x7f6a4c0091c0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 10 monclient: shutdown 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 msgr2=0x7f6a581f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009aa0 tx=0x7f6a400047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 shutdown_connections 2026-02-24T18:35:35.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a24078e50 0x7f6a2407b310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a580c99a0 0x7f6a581f5ea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a5815a5e0 0x7f6a581f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 --2- 10.20.193.18:0/4282241778 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a5806ff20 0x7f6a580c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 >> 10.20.193.18:0/4282241778 conn(0x7f6a580cd780 msgr2=0x7f6a580cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 shutdown_connections 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 -- 10.20.193.18:0/4282241778 wait complete. 2026-02-24T18:35:35.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.632+0000 7f6a5d0d3640 1 librados: shutdown 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14047030 con 0x7f2d18152bb0 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: dump: 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.656 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1400b2e0 con 0x7f2d18152bb0 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1403d040 con 0x7f2d18152bb0 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2cfeffd640 10 monclient: dump: 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.657 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d1c91b640 4 set_mon_vals no callback set 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 5 monclient: authenticate success, global_id 4502 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 10 monclient: _renew_subs 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d180cb7e0 con 0x7f2d18152bb0 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 10 client.4502.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 10 monclient: _renew_subs 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d180cbda0 con 0x7f2d18152bb0 2026-02-24T18:35:35.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f2d203a9640 1 librados: init done 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 10 monclient(hunting): handle_auth_done global_id 4495 payload 274 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 1 -- 10.20.193.18:0/2328711896 >> v1:10.20.193.110:6789/0 conn(0x7f15e41527f0 legacy=0x7f15e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 1 -- 10.20.193.18:0/2328711896 >> v1:10.20.193.32:6789/0 conn(0x7f15e4073530 legacy=0x7f15e415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.633+0000 7f15e9060640 1 -- 10.20.193.18:0/2328711896 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e4073c20 con 0x7f15e415fad0 2026-02-24T18:35:35.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.634+0000 7f2d203a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.634+0000 7f2d203a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.634+0000 7f2d203a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.634+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d180c5a80 con 0x7f2d18152bb0 2026-02-24T18:35:35.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.635+0000 7f15e9060640 1 --2- 10.20.193.18:0/2328711896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e415fad0 0x7f15e40ccf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009d90 tx=0x7f15e002f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502ddfdb26d11f25 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 1 -- 10.20.193.18:0/2328711896 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e003c070 con 0x7f15e415fad0 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: dump: 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 1 -- 10.20.193.18:0/2328711896 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e0037440 con 0x7f15e415fad0 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 1 -- 10.20.193.18:0/2328711896 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0037810 con 0x7f15e415fad0 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 5 monclient: authenticate success, global_id 4495 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 4 set_mon_vals no callback set 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15db7fe640 10 monclient: dump: 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 10 monclient: shutdown 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 1 -- 10.20.193.18:0/2328711896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e415fad0 msgr2=0x7f15e40ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.638+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2328711896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e415fad0 0x7f15e40ccf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009d90 tx=0x7f15e002f1e0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 -- 10.20.193.18:0/2328711896 shutdown_connections 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2328711896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e415fad0 0x7f15e40ccf70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 -- 10.20.193.18:0/2328711896 >> 10.20.193.18:0/2328711896 conn(0x7f15e40ce730 msgr2=0x7f15e40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 -- 10.20.193.18:0/2328711896 shutdown_connections 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 -- 10.20.193.18:0/2328711896 wait complete. 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 10 monclient: monmap: 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 librados: starting msgr at 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 librados: starting objecter 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 Processor -- start 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 -- start start 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 librados: setting wanted keys 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 1 librados: calling monclient init 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.639+0000 7f15eaaea640 10 monclient: init 2026-02-24T18:35:35.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f15e4073530 0x7f15e40c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-c con 0x7f15e4073530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-b con 0x7f15e41527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f15e415fad0 0x7f15e40c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): picked mon.noname-a con 0x7f15e415fad0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40744c0 con 0x7f15e415fad0 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40741e0 con 0x7f15e41527f0 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40747a0 con 0x7f15e4073530 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15eaaea640 10 monclient(hunting): authenticate will time out at 1479.321093s 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53470/0 (socket says 10.20.193.18:53470) 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 1 -- 10.20.193.18:0/2072914565 learned_addr learned my addr 10.20.193.18:0/2072914565 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): get_auth_request con 0x7f15e41527f0 auth_method 0 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.640+0000 7f15dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.0 2026-02-24T18:35:35.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 10 monclient(hunting): handle_auth_done global_id 4498 payload 824 2026-02-24T18:35:35.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 -- 10.20.193.18:0/2072914565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f15e4073530 msgr2=0x7f15e40c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f15e4073530 0x7f15e40c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 -- 10.20.193.18:0/2072914565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f15e415fad0 msgr2=0x7f15e40c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f15e415fad0 0x7f15e40c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.642+0000 7f15dbfff640 1 -- 10.20.193.18:0/2072914565 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e40c6f90 con 0x7f15e41527f0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.643+0000 7f15dbfff640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b570 tx=0x7f15cc00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.643+0000 7f15d97fa640 10 client.?.objecter ms_handle_connect 0x7f15e41527f0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.643+0000 7f15d97fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc013020 con 0x7f15e41527f0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: dump: 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.668 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc0044a0 con 0x7f15e41527f0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 5 monclient: authenticate success, global_id 4498 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 10 monclient: _renew_subs 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e40c7270 con 0x7f15e41527f0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0029c0 con 0x7f15e41527f0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15d97fa640 10 monclient: dump: 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.669 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15daffd640 4 set_mon_vals no callback set 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 10 client.4498.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 10 monclient: _renew_subs 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e40c7830 con 0x7f15e41527f0 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.644+0000 7f15eaaea640 1 librados: init done 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.645+0000 7f15eaaea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.645+0000 7f15eaaea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.645+0000 7f15eaaea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.645+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e4074fa0 con 0x7f15e41527f0 2026-02-24T18:35:35.672 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-24T18:35:35.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.673+0000 7f44effff640 1 -- 10.20.193.18:0/4094743582 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f450c0873a0 con 0x7f4518152b30 2026-02-24T18:35:35.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.673+0000 7f44effff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:35.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.673+0000 7f44effff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.674 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 1 -- 10.20.193.18:0/179638474 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc03c070 con 0x7efc080c99a0 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: dump: 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 1 -- 10.20.193.18:0/179638474 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc02fb20 con 0x7efc080c99a0 2026-02-24T18:35:35.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 5 monclient: authenticate success, global_id 4551 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 1 -- 10.20.193.18:0/179638474 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc037440 con 0x7efc080c99a0 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 4 set_mon_vals no callback set 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc0d1ca640 10 monclient: dump: 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 10 monclient: shutdown 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 -- 10.20.193.18:0/179638474 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc080c99a0 msgr2=0x7efc08070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 --2- 10.20.193.18:0/179638474 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc080c99a0 0x7efc08070300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009970 tx=0x7efbfc02eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 -- 10.20.193.18:0/179638474 shutdown_connections 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 --2- 10.20.193.18:0/179638474 >> v2:10.20.193.32:3300/0 conn(0x7efc0815a5e0 0x7efc0814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 --2- 10.20.193.18:0/179638474 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc080c99a0 0x7efc08070300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 -- 10.20.193.18:0/179638474 >> 10.20.193.18:0/179638474 conn(0x7efc080cd780 msgr2=0x7efc080cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 -- 10.20.193.18:0/179638474 shutdown_connections 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 1 -- 10.20.193.18:0/179638474 wait complete. 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.674+0000 7efc10457640 10 monclient: monmap: 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.678 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 librados: starting msgr at 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 librados: starting objecter 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 Processor -- start 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 -- start start 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 librados: setting wanted keys 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 librados: calling monclient init 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient: init 2026-02-24T18:35:35.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 10 librados: watch_flush enter 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 10 librados: watch_flush exit 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 20 client.4521.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 10 client.4521.objecter successfully canceled tick 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 msgr2=0x7f44e807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 0x7f44e807b310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4500006fd0 tx=0x7f4500006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 10 monclient: shutdown 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 msgr2=0x7f4518136720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f450c009ad0 tx=0x7f450c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 shutdown_connections 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f44e8078e50 0x7f44e807b310 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4518152f10 0x7f4518136c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f451815c730 0x7f45180c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 --2- 10.20.193.18:0/4094743582 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4518152b30 0x7f4518136720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 >> 10.20.193.18:0/4094743582 conn(0x7f4518074080 msgr2=0x7f45180c5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 shutdown_connections 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 -- 10.20.193.18:0/4094743582 wait complete. 2026-02-24T18:35:35.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7f451d911640 1 librados: shutdown 2026-02-24T18:35:35.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:35.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): picked mon.noname-a con 0x7efc0806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc080c99a0 0x7efc0815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): picked mon.noname-b con 0x7efc080c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efc0815a5e0 0x7efc0815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): picked mon.noname-c con 0x7efc0815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efc08150350 con 0x7efc0806ff20 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc080c8a10 con 0x7efc080c99a0 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc081504d0 con 0x7efc0815a5e0 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.675+0000 7efc10457640 10 monclient(hunting): authenticate will time out at 1479.356644s 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41680/0 (socket says 10.20.193.18:41680) 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 1 -- 10.20.193.18:0/2582402595 learned_addr learned my addr 10.20.193.18:0/2582402595 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 10 monclient(hunting): get_auth_request con 0x7efc0806ff20 auth_method 0 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.676+0000 7efc0e1cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.677+0000 7efc0e1cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.677+0000 7efc0e1cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.677+0000 7efc0e1cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 10 monclient(hunting): handle_auth_done global_id 4554 payload 824 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 -- 10.20.193.18:0/2582402595 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efc0815a5e0 msgr2=0x7efc0815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efc0815a5e0 0x7efc0815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 -- 10.20.193.18:0/2582402595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc080c99a0 msgr2=0x7efc0815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc080c99a0 0x7efc0815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.678+0000 7efc0e1cc640 1 -- 10.20.193.18:0/2582402595 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc081e6210 con 0x7efc0806ff20 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.679+0000 7efc0e1cc640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b570 tx=0x7efbf800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.679+0000 7efbeeffd640 10 client.?.objecter ms_handle_connect 0x7efc0806ff20 2026-02-24T18:35:35.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.679+0000 7efbeeffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8013020 con 0x7efc0806ff20 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: dump: 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf80044a0 con 0x7efc0806ff20 2026-02-24T18:35:35.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80029c0 con 0x7efc0806ff20 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efbeeffd640 10 monclient: dump: 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc0c9c9640 4 set_mon_vals no callback set 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 5 monclient: authenticate success, global_id 4554 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 10 monclient: _renew_subs 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc081e64f0 con 0x7efc0806ff20 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 10 client.4554.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 10 monclient: _renew_subs 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc081e6ab0 con 0x7efc0806ff20 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.681+0000 7efc10457640 1 librados: init done 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.682+0000 7efc10457640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.682+0000 7efc10457640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.682+0000 7efc10457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.682+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc081508a0 con 0x7efc0806ff20 2026-02-24T18:35:35.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.5 2026-02-24T18:35:35.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.694+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efbf8002b60 con 0x7efc0806ff20 2026-02-24T18:35:35.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.694+0000 7efbeeffd640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 0x7efbd807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.694+0000 7efc0d9cb640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 0x7efbd807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.694+0000 7efc0d9cb640 10 monclient: get_auth_request con 0x7efbd8078e50 auth_method 0 2026-02-24T18:35:35.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.695+0000 7efc0d9cb640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 0x7efbd807b310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc080709c0 tx=0x7efbfc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.695+0000 7efbeeffd640 10 client.4554.objecter ms_handle_connect 0x7efbd8078e50 2026-02-24T18:35:35.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.697+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7efbf805e100 con 0x7efc0806ff20 2026-02-24T18:35:35.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.697+0000 7efbeeffd640 10 client.4554.objecter ms_dispatch2 0x7efc08000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.697+0000 7efbeeffd640 3 client.4554.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.697+0000 7efbeeffd640 3 client.4554.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.697+0000 7efbeeffd640 20 client.4554.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.716+0000 7f5df7b53640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.716+0000 7f5df7b53640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.716+0000 7f5df7b53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.716+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5df0158ab0 con 0x7f5df015a5e0 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.724+0000 7f19e1217640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.724+0000 7f19e1217640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.724+0000 7f19e1217640 10 monclient: monmap: 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.725531+0000 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.725531+0000 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.727 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.727 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.727 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.727 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 Processor -- start 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 -- start start 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient: init 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient: _add_conns ranks=[4,3,1,2,5,0] 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f19dc155a50 0x7f19dc15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-b con 0x7f19dc155a50 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f19dc072e50 0x7f19dc0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-a con 0x7f19dc072e50 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f19dc0c9160 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19dc15fa90 con 0x7f19dc0c9160 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc160050 con 0x7f19dc072e50 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc15fd70 con 0x7f19dc155a50 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.725+0000 7f19e1217640 10 monclient(hunting): authenticate will time out at 1479.406694s 2026-02-24T18:35:35.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.742+0000 7f7d073ce640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.742+0000 7f7d073ce640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.742+0000 7f7d073ce640 10 monclient: monmap: 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.743408+0000 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.743408+0000 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.744 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.745 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.745 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.745 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.745 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 Processor -- start 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 -- start start 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient: init 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient: _add_conns ranks=[3,0,4,2,1,5] 2026-02-24T18:35:35.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7d001527f0 0x7f7d00073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-a con 0x7f7d001527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d0015d2c0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7d0015d730 0x7f7d000ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-b con 0x7f7d0015d730 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d00073c20 con 0x7f7d0015d2c0 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7d00074b10 con 0x7f7d001527f0 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d00073f00 con 0x7f7d0015d730 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.743+0000 7f7d073ce640 10 monclient(hunting): authenticate will time out at 1479.424781s 2026-02-24T18:35:35.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.748+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f15cc002b60 con 0x7f15e41527f0 2026-02-24T18:35:35.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.748+0000 7f15d97fa640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 0x7f15b407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.748+0000 7f15e885f640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 0x7f15b407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.748+0000 7f15e885f640 10 monclient: get_auth_request con 0x7f15b4078e30 auth_method 0 2026-02-24T18:35:35.750 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-24T18:35:35.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.749+0000 7f15d97fa640 10 client.4498.objecter ms_handle_connect 0x7f15b4078e30 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.749+0000 7f15e885f640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 0x7f15b407b2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15d4007fd0 tx=0x7f15d4007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.750+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f15cc05e060 con 0x7f15e41527f0 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.750+0000 7f15d97fa640 10 client.4498.objecter ms_dispatch2 0x7f15e4000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.750+0000 7f15d97fa640 3 client.4498.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.750+0000 7f15d97fa640 3 client.4498.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.750+0000 7f15d97fa640 20 client.4498.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.757+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f15cc01d020 con 0x7f15e41527f0 2026-02-24T18:35:35.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.757+0000 7f15d97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.757+0000 7f15d97fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.763+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2d14040030 con 0x7f2d18152bb0 2026-02-24T18:35:35.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.763+0000 7f2cfeffd640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 0x7f2ce807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.763+0000 7f2d1e11e640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 0x7f2ce807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.763+0000 7f2d1e11e640 10 monclient: get_auth_request con 0x7f2ce8078e80 auth_method 0 2026-02-24T18:35:35.765 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-24T18:35:35.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.764+0000 7f2d1e11e640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 0x7f2ce807b340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006fd0 tx=0x7f2d08006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.764+0000 7f2cfeffd640 10 client.4502.objecter ms_handle_connect 0x7f2ce8078e80 2026-02-24T18:35:35.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.765+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f2d1400b490 con 0x7f2d18152bb0 2026-02-24T18:35:35.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.765+0000 7f2cfeffd640 10 client.4502.objecter ms_dispatch2 0x7f2d18000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.765+0000 7f2cfeffd640 3 client.4502.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.765+0000 7f2cfeffd640 3 client.4502.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.765+0000 7f2cfeffd640 20 client.4502.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.767+0000 7f19dbfff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f19dc155a50 0x7f19dc15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.767+0000 7f7d05944640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7d0015d730 0x7f7d000ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7d0015d730 0x7f7d000ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53496/0 (socket says 10.20.193.18:53496) 2026-02-24T18:35:35.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 1 -- 10.20.193.18:0/835092996 learned_addr learned my addr 10.20.193.18:0/835092996 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): get_auth_request con 0x7f7d0015d730 auth_method 0 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19e0a16640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f19dc0c9160 0x7f19dc0c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51276/0 (socket says 10.20.193.18:51276) 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19e0a16640 1 -- 10.20.193.18:0/4263070639 learned_addr learned my addr 10.20.193.18:0/4263070639 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19dbfff640 10 monclient(hunting): get_auth_request con 0x7f19dc155a50 auth_method 0 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f19dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.768+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.769+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.769+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.769+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.769+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2d14051070 con 0x7f2d18152bb0 2026-02-24T18:35:35.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7f2cfeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7f2cfeffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590017070 con 0x7fe5a0156360 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 10 monclient: dump: 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.774 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.771+0000 7fe586ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59000ac40 con 0x7fe5a0156360 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590012400 con 0x7fe5a0156360 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe586ffd640 10 monclient: dump: 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe59cd72640 4 set_mon_vals no callback set 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 5 monclient: authenticate success, global_id 4505 2026-02-24T18:35:35.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _renew_subs 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a01f7f20 con 0x7fe5a0156360 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 client.4505.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _renew_subs 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01f41d0 con 0x7fe5a0156360 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 1 librados: init done 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.772+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a015e7a0 con 0x7fe5a0156360 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 10 monclient(hunting): handle_auth_done global_id 4504 payload 274 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 1 -- 10.20.193.18:0/835092996 >> v2:10.20.193.18:3300/0 conn(0x7f7d001527f0 msgr2=0x7f7d00073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 1 --2- 10.20.193.18:0/835092996 >> v2:10.20.193.18:3300/0 conn(0x7f7d001527f0 0x7f7d00073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 1 -- 10.20.193.18:0/835092996 >> v1:10.20.193.18:6789/0 conn(0x7f7d0015d2c0 legacy=0x7f7d0015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.795+0000 7f7d05944640 1 -- 10.20.193.18:0/835092996 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d000741e0 con 0x7f7d0015d730 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 10 monclient(hunting): handle_auth_done global_id 4501 payload 274 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 1 -- 10.20.193.18:0/4263070639 >> v2:10.20.193.18:3300/0 conn(0x7f19dc072e50 msgr2=0x7f19dc0732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 1 --2- 10.20.193.18:0/4263070639 >> v2:10.20.193.18:3300/0 conn(0x7f19dc072e50 0x7f19dc0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 1 -- 10.20.193.18:0/4263070639 >> v1:10.20.193.32:6789/0 conn(0x7f19dc0c9160 legacy=0x7f19dc0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.802+0000 7f19dbfff640 1 -- 10.20.193.18:0/4263070639 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc072370 con 0x7f19dc155a50 2026-02-24T18:35:35.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.805+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efbf801d020 con 0x7efc0806ff20 2026-02-24T18:35:35.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.805+0000 7efbeeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.805+0000 7efbeeffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.808+0000 7f19dbfff640 1 --2- 10.20.193.18:0/4263070639 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc155a50 0x7f19dc15f150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19dc071970 tx=0x7f19c802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8deb588b745957f1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.809+0000 7f7d05944640 1 --2- 10.20.193.18:0/835092996 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000ccf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0006cdf0 tx=0x7f7cfc02f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5957487e7d55124e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f5dde7fc640 1 -- 10.20.193.18:0/2748690341 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5dec065f10 con 0x7f5df015a5e0 2026-02-24T18:35:35.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f5dde7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:35.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f5dde7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 1 -- 10.20.193.18:0/835092996 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc02fce0 con 0x7f7d0015d730 2026-02-24T18:35:35.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 10 monclient: dump: 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7ceffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7d073ce640 5 monclient: authenticate success, global_id 4504 2026-02-24T18:35:35.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.812+0000 7f7d073ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:35.816 INFO:teuthology.orchestra.run.trial018.stdout:90194313218 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 10 librados: watch_flush enter 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 10 librados: watch_flush exit 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 20 client.4492.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 10 client.4492.objecter successfully canceled tick 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 msgr2=0x7f5dc007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 0x7f5dc007b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5de0039000 tx=0x7f5de0033000 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 10 monclient: shutdown 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 msgr2=0x7f5df01f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00cb10 tx=0x7f5dec00a060 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 shutdown_connections 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5dc0078e00 0x7f5dc007b2c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5df015a5e0 0x7f5df01f5db0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f5df00c99a0 0x7f5df00c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 --2- 10.20.193.18:0/2748690341 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5df006ff20 0x7f5df01f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 >> 10.20.193.18:0/2748690341 conn(0x7f5df00cd780 msgr2=0x7f5df006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 shutdown_connections 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.814+0000 7f5df7b53640 1 -- 10.20.193.18:0/2748690341 wait complete. 2026-02-24T18:35:35.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.815+0000 7f5df7b53640 1 librados: shutdown 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 1 -- 10.20.193.18:0/835092996 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0373f0 con 0x7f7d0015d730 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 1 -- 10.20.193.18:0/835092996 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0377c0 con 0x7f7d0015d730 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7ceffff640 10 monclient: dump: 2026-02-24T18:35:35.819 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7d073ce640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f7d073ce640 4 set_mon_vals no callback set 2026-02-24T18:35:35.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 1 -- 10.20.193.18:0/4263070639 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c803c070 con 0x7f19dc155a50 2026-02-24T18:35:35.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:35.821 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: dump: 2026-02-24T18:35:35.821 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 1 -- 10.20.193.18:0/4263070639 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c802faf0 con 0x7f19dc155a50 2026-02-24T18:35:35.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19e1217640 5 monclient: authenticate success, global_id 4501 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 1 -- 10.20.193.18:0/4263070639 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8037440 con 0x7f19dc155a50 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19e1217640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19e1217640 4 set_mon_vals no callback set 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.816+0000 7f19daffd640 10 monclient: dump: 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.823 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.824 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.824 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.824 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 10 monclient: shutdown 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 -- 10.20.193.18:0/835092996 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 msgr2=0x7f7d000ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 --2- 10.20.193.18:0/835092996 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000ccf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0006cdf0 tx=0x7f7cfc02f280 comp rx=0 tx=0).stop 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 -- 10.20.193.18:0/835092996 shutdown_connections 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 --2- 10.20.193.18:0/835092996 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000ccf70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 --2- 10.20.193.18:0/835092996 >> v2:10.20.193.18:3300/0 conn(0x7f7d001527f0 0x7f7d00073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 -- 10.20.193.18:0/835092996 >> 10.20.193.18:0/835092996 conn(0x7f7d000ce730 msgr2=0x7f7d000ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 -- 10.20.193.18:0/835092996 shutdown_connections 2026-02-24T18:35:35.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 -- 10.20.193.18:0/835092996 wait complete. 2026-02-24T18:35:35.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 10 monclient: shutdown 2026-02-24T18:35:35.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 -- 10.20.193.18:0/4263070639 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc155a50 msgr2=0x7f19dc15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 --2- 10.20.193.18:0/4263070639 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc155a50 0x7f19dc15f150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19dc071970 tx=0x7f19c802ede0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 -- 10.20.193.18:0/4263070639 shutdown_connections 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 --2- 10.20.193.18:0/4263070639 >> v2:10.20.193.18:3300/0 conn(0x7f19dc072e50 0x7f19dc0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 --2- 10.20.193.18:0/4263070639 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc155a50 0x7f19dc15f150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 -- 10.20.193.18:0/4263070639 >> 10.20.193.18:0/4263070639 conn(0x7f19dc0cd480 msgr2=0x7f19dc0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 -- 10.20.193.18:0/4263070639 shutdown_connections 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 1 -- 10.20.193.18:0/4263070639 wait complete. 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 10 monclient: monmap: 2026-02-24T18:35:35.827 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.828 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f19e1217640 10 monclient: monmap: 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.829 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 librados: starting msgr at 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.817+0000 7f7d073ce640 1 librados: starting objecter 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 Processor -- start 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 -- start start 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 librados: setting wanted keys 2026-02-24T18:35:35.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 librados: calling monclient init 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient: init 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 librados: starting msgr at 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 librados: starting objecter 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 Processor -- start 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 -- start start 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 librados: setting wanted keys 2026-02-24T18:35:35.831 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 librados: calling monclient init 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient: init 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 10 monclient: monmap: 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.818983+0000 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.818983+0000 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.832 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.833 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d001527f0 0x7f7d000c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-c con 0x7f7d001527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d0015d2c0 0x7f7d001f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-a con 0x7f7d0015d2c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): picked mon.noname-b con 0x7f7d0015d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00074fa0 con 0x7f7d0015d2c0 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d000747a0 con 0x7f7d0015d730 2026-02-24T18:35:35.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00075120 con 0x7f7d001527f0 2026-02-24T18:35:35.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f7d073ce640 10 monclient(hunting): authenticate will time out at 1479.499531s 2026-02-24T18:35:35.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 1 Processor -- start 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-b con 0x7f19dc072e50 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f19dc0c9160 0x7f19dc1f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-c con 0x7f19dc0c9160 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f19dc155a50 0x7f19dc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): picked mon.noname-a con 0x7f19dc155a50 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc155e30 con 0x7f19dc155a50 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc072930 con 0x7f19dc072e50 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc155fb0 con 0x7f19dc0c9160 2026-02-24T18:35:35.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f19e1217640 10 monclient(hunting): authenticate will time out at 1479.499647s 2026-02-24T18:35:35.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 1 -- start start 2026-02-24T18:35:35.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.818+0000 7f0e82387640 10 monclient: init 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient: _add_conns ranks=[1,0,3,2,5,4] 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0e7c06ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e7c155eb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0e7c15c970 0x7f0e7c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-a con 0x7f0e7c15c970 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7c0725b0 con 0x7f0e7c155eb0 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7c072890 con 0x7f0e7c06ff20 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7c072b70 con 0x7f0e7c15c970 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.819+0000 7f0e82387640 10 monclient(hunting): authenticate will time out at 1479.500257s 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53516/0 (socket says 10.20.193.18:53516) 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 1 -- 10.20.193.18:0/3124514010 learned_addr learned my addr 10.20.193.18:0/3124514010 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 10 monclient(hunting): get_auth_request con 0x7f19dc072e50 auth_method 0 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.820+0000 7f19dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.6 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.824+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.824+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.824+0000 7f19dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 10 monclient(hunting): handle_auth_done global_id 4513 payload 824 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 -- 10.20.193.18:0/3124514010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f19dc0c9160 msgr2=0x7f19dc1f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f19dc0c9160 0x7f19dc1f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 -- 10.20.193.18:0/3124514010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f19dc155a50 msgr2=0x7f19dc06e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f19dc155a50 0x7f19dc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.826+0000 7f19dbfff640 1 -- 10.20.193.18:0/3124514010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc157b00 con 0x7f19dc072e50 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.832+0000 7f19dbfff640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19c802f310 tx=0x7f19c80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.832+0000 7f19d8ff9640 10 client.?.objecter ms_handle_connect 0x7f19dc072e50 2026-02-24T18:35:35.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.832+0000 7f19d8ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.833+0000 7fa0f3914640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.833+0000 7fa0f3914640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.833+0000 7fa0f3914640 10 monclient: monmap: 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.834528+0000 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.834528+0000 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.843 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 Processor -- start 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 -- start start 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient: init 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient: _add_conns ranks=[3,4,2,1,5,0] 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa0ec155e70 0x7fa0ec074120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-a con 0x7fa0ec155e70 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa0ec074d60 0x7fa0ec0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-b con 0x7fa0ec074d60 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa0ec156250 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0ec0ccb50 con 0x7fa0ec156250 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0cce30 con 0x7fa0ec155e70 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0cd110 con 0x7fa0ec074d60 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.834+0000 7fa0f3914640 10 monclient(hunting): authenticate will time out at 1479.515731s 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53504/0 (socket says 10.20.193.18:53504) 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 1 -- 10.20.193.18:0/2085884724 learned_addr learned my addr 10.20.193.18:0/2085884724 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 10 monclient(hunting): get_auth_request con 0x7f7d0015d730 auth_method 0 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f7d05944640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f0e7bfff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f0e7c06ff20 0x7f0e7c1557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51292/0 (socket says 10.20.193.18:51292) 2026-02-24T18:35:35.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.841+0000 7f0e7bfff640 1 -- 10.20.193.18:0/165834222 learned_addr learned my addr 10.20.193.18:0/165834222 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.846 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-24T18:35:35.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.842+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.842+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.842+0000 7f7d05944640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.842+0000 7fa0f0e88640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa0ec074d60 0x7fa0ec0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 10 monclient(hunting): handle_auth_done global_id 4516 payload 824 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 -- 10.20.193.18:0/2085884724 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d001527f0 msgr2=0x7f7d000c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d001527f0 0x7f7d000c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 -- 10.20.193.18:0/2085884724 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d0015d2c0 msgr2=0x7f7d001f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d0015d2c0 0x7f7d001f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.843+0000 7f7d05944640 1 -- 10.20.193.18:0/2085884724 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d000c7130 con 0x7f7d0015d730 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa0ec074d60 0x7fa0ec0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53518/0 (socket says 10.20.193.18:53518) 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 1 -- 10.20.193.18:0/585681122 learned_addr learned my addr 10.20.193.18:0/585681122 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 10 monclient(hunting): get_auth_request con 0x7fa0ec074d60 auth_method 0 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7fa0f0e88640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7f15eaaea640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7f15eaaea640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7f15eaaea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.844+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f15e415eca0 con 0x7f15e41527f0 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.845+0000 7fa0f0e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.845+0000 7fa0f0e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.845+0000 7fa0f0e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.846+0000 7f7d05944640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc02f2c0 tx=0x7f7cfc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.846+0000 7f7cedffb640 10 client.?.objecter ms_handle_connect 0x7f7d0015d730 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.846+0000 7f7cedffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 10 monclient(hunting): handle_auth_done global_id 4519 payload 274 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 1 -- 10.20.193.18:0/585681122 >> v2:10.20.193.18:3300/0 conn(0x7fa0ec155e70 msgr2=0x7fa0ec074120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 1 --2- 10.20.193.18:0/585681122 >> v2:10.20.193.18:3300/0 conn(0x7fa0ec155e70 0x7fa0ec074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 1 -- 10.20.193.18:0/585681122 >> v1:10.20.193.110:6789/0 conn(0x7fa0ec156250 legacy=0x7fa0ec15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.847+0000 7fa0f0e88640 1 -- 10.20.193.18:0/585681122 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0cd3f0 con 0x7fa0ec074d60 2026-02-24T18:35:35.851 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-24T18:35:35.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.849+0000 7fa0f0e88640 1 --2- 10.20.193.18:0/585681122 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0cc210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec08f620 tx=0x7fa0e002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6646cf4bd07b84 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.850+0000 7f0e808fd640 1 --2- 10.20.193.18:0/165834222 >> v2:10.20.193.18:3300/0 conn(0x7f0e7c15c970 0x7f0e7c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): get_auth_request con 0x7f0e7c15c970 auth_method 0 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.851+0000 7f0e808fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 10 monclient(hunting): handle_auth_done global_id 4557 payload 274 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 1 -- 10.20.193.18:0/165834222 >> v1:10.20.193.32:6789/0 conn(0x7f0e7c06ff20 legacy=0x7f0e7c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 1 -- 10.20.193.18:0/165834222 >> v1:10.20.193.18:6789/0 conn(0x7f0e7c155eb0 legacy=0x7f0e7c0c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.853+0000 7f0e808fd640 1 -- 10.20.193.18:0/165834222 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c072e50 con 0x7f0e7c15c970 2026-02-24T18:35:35.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.855+0000 7f0e808fd640 1 --2- 10.20.193.18:0/165834222 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c15c970 0x7f0e7c070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009d90 tx=0x7f0e7002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a9c79eec3bc597 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 1 -- 10.20.193.18:0/165834222 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7003c070 con 0x7f0e7c15c970 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: dump: 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 1 -- 10.20.193.18:0/165834222 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e70037440 con 0x7f0e7c15c970 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 1 -- 10.20.193.18:0/165834222 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70037810 con 0x7f0e7c15c970 2026-02-24T18:35:35.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 5 monclient: authenticate success, global_id 4557 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 4 set_mon_vals no callback set 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e7affd640 10 monclient: dump: 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.861 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 10 monclient: shutdown 2026-02-24T18:35:35.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 -- 10.20.193.18:0/165834222 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c15c970 msgr2=0x7f0e7c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 --2- 10.20.193.18:0/165834222 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c15c970 0x7f0e7c070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009d90 tx=0x7f0e7002f1e0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 -- 10.20.193.18:0/165834222 shutdown_connections 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 --2- 10.20.193.18:0/165834222 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c15c970 0x7f0e7c070300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 -- 10.20.193.18:0/165834222 >> 10.20.193.18:0/165834222 conn(0x7f0e7c0cd780 msgr2=0x7f0e7c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 -- 10.20.193.18:0/165834222 shutdown_connections 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.858+0000 7f0e82387640 1 -- 10.20.193.18:0/165834222 wait complete. 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 10 monclient: monmap: 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 librados: starting msgr at 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 librados: starting objecter 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 Processor -- start 2026-02-24T18:35:35.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 -- start start 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 librados: setting wanted keys 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 1 librados: calling monclient init 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.859+0000 7f0e82387640 10 monclient: init 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-a con 0x7f0e7c06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0e7c155eb0 0x7f0e7c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-b con 0x7f0e7c155eb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0e7c15c970 0x7f0e7c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): picked mon.noname-c con 0x7f0e7c15c970 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c15d340 con 0x7f0e7c06ff20 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c15d1c0 con 0x7f0e7c155eb0 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c15d4c0 con 0x7f0e7c15c970 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e82387640 10 monclient(hunting): authenticate will time out at 1479.541129s 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41764/0 (socket says 10.20.193.18:41764) 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 1 -- 10.20.193.18:0/1980846389 learned_addr learned my addr 10.20.193.18:0/1980846389 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): get_auth_request con 0x7f0e7c06ff20 auth_method 0 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.860+0000 7f0e7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.861+0000 7f2d203a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.861+0000 7f2d203a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.861+0000 7f2d203a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.861+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2d180c7340 con 0x7f2d18152bb0 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7b7fe640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0e7c155eb0 0x7f0e7c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7b7fe640 10 monclient(hunting): get_auth_request con 0x7f0e7c155eb0 auth_method 0 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 10 monclient(hunting): handle_auth_done global_id 4560 payload 824 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 -- 10.20.193.18:0/1980846389 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0e7c15c970 msgr2=0x7f0e7c1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0e7c15c970 0x7f0e7c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 -- 10.20.193.18:0/1980846389 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0e7c155eb0 msgr2=0x7f0e7c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0e7c155eb0 0x7f0e7c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 monclient: found mon.noname-a 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.862+0000 7f0e7bfff640 1 -- 10.20.193.18:0/1980846389 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1f3be0 con 0x7f0e7c06ff20 2026-02-24T18:35:35.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.864+0000 7f0e7bfff640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c730 tx=0x7f0e6800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.864+0000 7f0e78ff9640 10 client.?.objecter ms_handle_connect 0x7f0e7c06ff20 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.864+0000 7f0e78ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68017070 con 0x7f0e7c06ff20 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: dump: 2026-02-24T18:35:35.867 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e68005bb0 con 0x7f0e7c06ff20 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68007070 con 0x7f0e7c06ff20 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 5 monclient: authenticate success, global_id 4560 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _renew_subs 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1f3ec0 con 0x7f0e7c06ff20 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e7a7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e78ff9640 10 monclient: dump: 2026-02-24T18:35:35.868 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 client.4560.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _renew_subs 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c153bf0 con 0x7f0e7c06ff20 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 1 librados: init done 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.866+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c15c750 con 0x7f0e7c06ff20 2026-02-24T18:35:35.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.878+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0e68007210 con 0x7f0e7c06ff20 2026-02-24T18:35:35.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.879+0000 7f0e78ff9640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 0x7f0e4c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.879+0000 7f0e7b7fe640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 0x7f0e4c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.879+0000 7f0e7b7fe640 10 monclient: get_auth_request con 0x7f0e4c078e00 auth_method 0 2026-02-24T18:35:35.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.879+0000 7f0e78ff9640 10 client.4560.objecter ms_handle_connect 0x7f0e4c078e00 2026-02-24T18:35:35.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.879+0000 7f0e7b7fe640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 0x7f0e4c07b2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006fd0 tx=0x7f0e6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.881+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0e6805d580 con 0x7f0e7c06ff20 2026-02-24T18:35:35.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.881+0000 7f0e78ff9640 10 client.4560.objecter ms_dispatch2 0x7f0e7c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.881+0000 7f0e78ff9640 3 client.4560.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.881+0000 7f0e78ff9640 3 client.4560.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.881+0000 7f0e78ff9640 20 client.4560.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.884+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe5900125a0 con 0x7fe5a0156360 2026-02-24T18:35:35.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe586ffd640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe59e575640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe59e575640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 msgr2=0x7fe56c07b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2026-02-24T18:35:35.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe59e575640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 msgr2=0x7fe56c07b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-24T18:35:35.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe59e575640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:35:35.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.885+0000 7fe59e575640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:35:35.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.887+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fe59005dc20 con 0x7fe5a0156360 2026-02-24T18:35:35.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.887+0000 7fe586ffd640 10 client.4505.objecter ms_dispatch2 0x7fe5a0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.887+0000 7fe586ffd640 3 client.4505.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.887+0000 7fe586ffd640 3 client.4505.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.887+0000 7fe586ffd640 20 client.4505.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.893+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe5a015e7a0 con 0x7fe5a0156360 2026-02-24T18:35:35.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.893+0000 7fe586ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.893+0000 7fe586ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.896+0000 7efc10457640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:35.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.896+0000 7efc10457640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:35.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.896+0000 7efc10457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:35.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.896+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7efc080cdee0 con 0x7efc0806ff20 2026-02-24T18:35:35.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.900+0000 7f2cfeffd640 1 -- 10.20.193.18:0/1007822029 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d140880f0 con 0x7f2d18152bb0 2026-02-24T18:35:35.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.900+0000 7f2cfeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:35.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.900+0000 7f2cfeffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.901 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:35.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 10 librados: watch_flush enter 2026-02-24T18:35:35.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 10 librados: watch_flush exit 2026-02-24T18:35:35.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 20 client.4502.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 10 client.4502.objecter successfully canceled tick 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 msgr2=0x7f2ce807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 0x7f2ce807b340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006fd0 tx=0x7f2d08006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 10 monclient: shutdown 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 msgr2=0x7f2d181f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d250 tx=0x7f2d14004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 shutdown_connections 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2ce8078e80 0x7f2ce807b340 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d181527d0 0x7f2d181f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d18152bb0 0x7f2d181f4330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 --2- 10.20.193.18:0/1007822029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d180c6190 0x7f2d180cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 >> 10.20.193.18:0/1007822029 conn(0x7f2d180cd780 msgr2=0x7f2d181471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 shutdown_connections 2026-02-24T18:35:35.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 -- 10.20.193.18:0/1007822029 wait complete. 2026-02-24T18:35:35.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.902+0000 7f2d203a9640 1 librados: shutdown 2026-02-24T18:35:35.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.3 2026-02-24T18:35:35.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.918+0000 7f0703c96640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.918+0000 7f0703c96640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.918+0000 7f0703c96640 10 monclient: monmap: 2026-02-24T18:35:35.920 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.920 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.919813+0000 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.919813+0000 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.921 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 Processor -- start 2026-02-24T18:35:35.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 -- start start 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient: init 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient: _add_conns ranks=[4,1,3,2,5,0] 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f06fc06ff20 0x7f06fc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-b con 0x7f06fc06ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f06fc0c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f06fc15a5e0 0x7f06fc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-a con 0x7f06fc15a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06fc0c7e90 con 0x7f06fc0c99a0 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f06fc159a40 con 0x7f06fc15a5e0 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.919+0000 7f0703c96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0703c96640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f06fc0c8170 con 0x7f06fc06ff20 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0703c96640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0703c96640 10 monclient(hunting): authenticate will time out at 1479.601012s 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 10 monclient: get_monmap_and_config 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 10 monclient: monmap: 2026-02-24T18:35:35.924 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:35.921345+0000 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:35.921345+0000 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:35.925 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0701a0b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f06fc06ff20 0x7f06fc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f070120a640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f06fc0c99a0 0x7f06fc159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51308/0 (socket says 10.20.193.18:51308) 2026-02-24T18:35:35.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f070120a640 1 -- 10.20.193.18:0/4206584613 learned_addr learned my addr 10.20.193.18:0/4206584613 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:35.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 1 Processor -- start 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 1 -- start start 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.920+0000 7f0b4ccea640 10 monclient: init 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): get_auth_request con 0x7f06fc06ff20 auth_method 0 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient: _add_conns ranks=[4,5,3,0,1,2] 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0b48155c70 0x7f0b4815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-b con 0x7f0b48155c70 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b480ccb30 0x7f0b4815d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-c con 0x7f0b480ccb30 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0b48156050 0x7f0b4815be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-a con 0x7f0b48156050 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0b4806f440 con 0x7f0b48156050 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0b48072e90 con 0x7f0b48155c70 2026-02-24T18:35:35.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480cbcb0 con 0x7f0b480ccb30 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b4ccea640 10 monclient(hunting): authenticate will time out at 1479.602502s 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0701a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.921+0000 7f0b45d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b480ccb30 0x7f0b4815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0b480ccb30 0x7f0b4815d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44510/0 (socket says 10.20.193.18:44510) 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 1 -- 10.20.193.18:0/2732188110 learned_addr learned my addr 10.20.193.18:0/2732188110 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): get_auth_request con 0x7f0b480ccb30 auth_method 0 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.922+0000 7f0b45d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 10 monclient(hunting): handle_auth_done global_id 4508 payload 274 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 -- 10.20.193.18:0/2732188110 >> v2:10.20.193.32:3300/0 conn(0x7f0b48155c70 msgr2=0x7f0b4815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 --2- 10.20.193.18:0/2732188110 >> v2:10.20.193.32:3300/0 conn(0x7f0b48155c70 0x7f0b4815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 -- 10.20.193.18:0/2732188110 >> v2:10.20.193.18:3300/0 conn(0x7f0b48156050 msgr2=0x7f0b4815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 --2- 10.20.193.18:0/2732188110 >> v2:10.20.193.18:3300/0 conn(0x7f0b48156050 0x7f0b4815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 monclient: found mon.noname-c 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.923+0000 7f0b45d74640 1 -- 10.20.193.18:0/2732188110 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4806f690 con 0x7f0b480ccb30 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 10 monclient(hunting): handle_auth_done global_id 4525 payload 274 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 1 -- 10.20.193.18:0/4206584613 >> v2:10.20.193.18:3300/0 conn(0x7f06fc15a5e0 msgr2=0x7f06fc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 1 --2- 10.20.193.18:0/4206584613 >> v2:10.20.193.18:3300/0 conn(0x7f06fc15a5e0 0x7f06fc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 1 -- 10.20.193.18:0/4206584613 >> v1:10.20.193.32:6789/0 conn(0x7f06fc0c99a0 legacy=0x7f06fc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 1 monclient: found mon.noname-b 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0701a0b640 1 -- 10.20.193.18:0/4206584613 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc0c8450 con 0x7f06fc06ff20 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.924+0000 7f0b45d74640 1 --2- 10.20.193.18:0/2732188110 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b4815d800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b48076d30 tx=0x7f0b3002ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32e5711e1f6dab9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 1 -- 10.20.193.18:0/2732188110 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3003c070 con 0x7f0b480ccb30 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: dump: 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.936 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 5 monclient: authenticate success, global_id 4508 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 1 -- 10.20.193.18:0/2732188110 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3002fbd0 con 0x7f0b480ccb30 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 1 -- 10.20.193.18:0/2732188110 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30037440 con 0x7f0b480ccb30 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 4 set_mon_vals no callback set 2026-02-24T18:35:35.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b45573640 10 monclient: dump: 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 10 monclient: shutdown 2026-02-24T18:35:35.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 -- 10.20.193.18:0/2732188110 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 msgr2=0x7f0b4815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/2732188110 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b4815d800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b48076d30 tx=0x7f0b3002ef50 comp rx=0 tx=0).stop 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 -- 10.20.193.18:0/2732188110 shutdown_connections 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/2732188110 >> v2:10.20.193.18:3300/0 conn(0x7f0b48156050 0x7f0b4815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/2732188110 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b4815d800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/2732188110 >> v2:10.20.193.32:3300/0 conn(0x7f0b48155c70 0x7f0b4815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 -- 10.20.193.18:0/2732188110 >> 10.20.193.18:0/2732188110 conn(0x7f0b480cc010 msgr2=0x7f0b480cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 -- 10.20.193.18:0/2732188110 shutdown_connections 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.926+0000 7f0b4ccea640 1 -- 10.20.193.18:0/2732188110 wait complete. 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient: build_initial_monmap 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient: monmap: 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:35.939 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0701a0b640 1 --2- 10.20.193.18:0/4206584613 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc06ff20 0x7f06fc070300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0098a0 tx=0x7f06ec02ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb6f1567e626e41 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 librados: starting msgr at 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 librados: starting objecter 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 Processor -- start 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 -- start start 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 librados: setting wanted keys 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 librados: calling monclient init 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient: init 2026-02-24T18:35:35.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-c con 0x7f0b480ccb30 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b48155c70 0x7f0b480c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-a con 0x7f0b48155c70 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b48156050 0x7f0b480c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): picked mon.noname-b con 0x7f0b48156050 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4806ff30 con 0x7f0b48155c70 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4806fc50 con 0x7f0b48156050 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b48070210 con 0x7f0b480ccb30 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.927+0000 7f0b4ccea640 10 monclient(hunting): authenticate will time out at 1479.608798s 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44516/0 (socket says 10.20.193.18:44516) 2026-02-24T18:35:35.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 1 -- 10.20.193.18:0/555065449 learned_addr learned my addr 10.20.193.18:0/555065449 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): get_auth_request con 0x7f0b480ccb30 auth_method 0 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46d76640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b48156050 0x7f0b480c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.928+0000 7f0b46575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): get_auth_request con 0x7f0b48156050 auth_method 0 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 10 monclient(hunting): handle_auth_done global_id 4511 payload 824 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 -- 10.20.193.18:0/555065449 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b48156050 msgr2=0x7f0b480c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b48156050 0x7f0b480c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 -- 10.20.193.18:0/555065449 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b48155c70 msgr2=0x7f0b480c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b48155c70 0x7f0b480c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 monclient: found mon.noname-c 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.929+0000 7f0b46575640 1 -- 10.20.193.18:0/555065449 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b481f4140 con 0x7f0b480ccb30 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.930+0000 7f0b46575640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b500 tx=0x7f0b3c00b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.930+0000 7f0b2effd640 10 client.?.objecter ms_handle_connect 0x7f0b480ccb30 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.930+0000 7f0b2effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c004480 con 0x7f0b480ccb30 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: dump: 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.946 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: _finish_auth 0 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c014070 con 0x7f0b480ccb30 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 5 monclient: authenticate success, global_id 4511 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _renew_subs 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b481f4420 con 0x7f0b480ccb30 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c002920 con 0x7f0b480ccb30 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b2effd640 10 monclient: dump: 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:35.947 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b44d72640 4 set_mon_vals no callback set 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 client.4511.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _renew_subs 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481f49e0 con 0x7f0b480ccb30 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 1 librados: init done 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.932+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b48074850 con 0x7f0b480ccb30 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.944+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0b3c002ac0 con 0x7f0b480ccb30 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.944+0000 7f0b2effd640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 0x7f0b1407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b45d74640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 0x7f0b1407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b45d74640 10 monclient: get_auth_request con 0x7f0b14078e00 auth_method 0 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b45d74640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 0x7f0b1407b2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0b48069a40 tx=0x7f0b30031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 10 client.4511.objecter ms_handle_connect 0x7f0b14078e00 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0b3c05de00 con 0x7f0b480ccb30 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 10 client.4511.objecter ms_dispatch2 0x7f0b48000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 3 client.4511.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 3 client.4511.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:35.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.945+0000 7f0b2effd640 20 client.4511.objecter dump_active .. 0 homeless 2026-02-24T18:35:35.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.978+0000 7fe5a4c98640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:35.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.978+0000 7fe5a4c98640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:35.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.978+0000 7fe5a4c98640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:35.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.978+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7fe5a015dbf0 con 0x7fe5a0156360 2026-02-24T18:35:35.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.986+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0e68061540 con 0x7f0e7c06ff20 2026-02-24T18:35:35.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.986+0000 7f0e78ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:35.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.986+0000 7f0e78ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:35.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.993+0000 7efbeeffd640 1 -- 10.20.193.18:0/2582402595 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7efbf80627c0 con 0x7efc0806ff20 2026-02-24T18:35:35.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.993+0000 7efbeeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:35.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.993+0000 7efbeeffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:35.994 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 10 librados: watch_flush enter 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 10 librados: watch_flush exit 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 20 client.4554.objecter shutdown clearing up homeless session... 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 10 client.4554.objecter successfully canceled tick 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 msgr2=0x7efbd807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 0x7efbd807b310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efc080709c0 tx=0x7efbfc031000 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 10 monclient: shutdown 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 msgr2=0x7efc0814c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b570 tx=0x7efbf800ba40 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 shutdown_connections 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efbd8078e50 0x7efbd807b310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efc080c99a0 0x7efc0815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efc0815a5e0 0x7efc0815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 --2- 10.20.193.18:0/2582402595 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efc0806ff20 0x7efc0814c3d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 >> 10.20.193.18:0/2582402595 conn(0x7efc080cd780 msgr2=0x7efc0814e400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 shutdown_connections 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 -- 10.20.193.18:0/2582402595 wait complete. 2026-02-24T18:35:35.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:35.995+0000 7efc10457640 1 librados: shutdown 2026-02-24T18:35:36.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.11 2026-02-24T18:35:36.014 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:35:36.014 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-24T18:35:36.044 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-24T18:35:36.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.045+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0b3c0624c0 con 0x7f0b480ccb30 2026-02-24T18:35:36.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.045+0000 7f0b2effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.045+0000 7f0b2effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.051 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.048+0000 7fe586ffd640 1 -- 10.20.193.18:0/605673827 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fe590022020 con 0x7fe5a0156360 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.048+0000 7fe586ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.048+0000 7fe586ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 10 librados: watch_flush enter 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 10 librados: watch_flush exit 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 20 client.4505.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 10 client.4505.objecter successfully canceled tick 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 msgr2=0x7fe56c07b310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 10 monclient: shutdown 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 msgr2=0x7fe5a0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ca80 tx=0x7fe59000cf50 comp rx=0 tx=0).stop 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 shutdown_connections 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe56c078e50 0x7fe56c07b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5a0155f80 0x7fe5a00c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5a0156360 0x7fe5a0158110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 --2- 10.20.193.18:0/605673827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5a00ccb40 0x7fe5a01567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 >> 10.20.193.18:0/605673827 conn(0x7fe5a00cc020 msgr2=0x7fe5a00cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.050+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 shutdown_connections 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.051+0000 7fe5a4c98640 1 -- 10.20.193.18:0/605673827 wait complete. 2026-02-24T18:35:36.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.051+0000 7fe5a4c98640 1 librados: shutdown 2026-02-24T18:35:36.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.14 2026-02-24T18:35:36.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.072+0000 7f0e82387640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:36.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.072+0000 7f0e82387640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:36.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.072+0000 7f0e82387640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.072+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f0e7c15db10 con 0x7f0e7c06ff20 2026-02-24T18:35:36.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.075+0000 7f0e78ff9640 1 -- 10.20.193.18:0/1980846389 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f0e68065840 con 0x7f0e7c06ff20 2026-02-24T18:35:36.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.075+0000 7f0e78ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:36.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.075+0000 7f0e78ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.077 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:36.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 10 librados: watch_flush enter 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 10 librados: watch_flush exit 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 20 client.4560.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 10 client.4560.objecter successfully canceled tick 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 msgr2=0x7f0e4c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 0x7f0e4c07b2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006fd0 tx=0x7f0e6c008040 comp rx=0 tx=0).stop 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 10 monclient: shutdown 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 msgr2=0x7f0e7c1f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c730 tx=0x7f0e6800cc00 comp rx=0 tx=0).stop 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 shutdown_connections 2026-02-24T18:35:36.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0e4c078e00 0x7f0e4c07b2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0e7c155eb0 0x7f0e7c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0e7c15c970 0x7f0e7c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 --2- 10.20.193.18:0/1980846389 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0e7c06ff20 0x7f0e7c1f3480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 >> 10.20.193.18:0/1980846389 conn(0x7f0e7c0cd780 msgr2=0x7f0e7c14fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 shutdown_connections 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 -- 10.20.193.18:0/1980846389 wait complete. 2026-02-24T18:35:36.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.077+0000 7f0e82387640 1 librados: shutdown 2026-02-24T18:35:36.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.13 2026-02-24T18:35:36.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 10 monclient: monmap: 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.115218+0000 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.115218+0000 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.116 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fe62baea640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fe62baea640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fe62baea640 10 monclient: monmap: 2026-02-24T18:35:36.117 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.115449+0000 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.115449+0000 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.118 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7f2955714640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7f2955714640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7f2955714640 10 monclient: monmap: 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.115640+0000 2026-02-24T18:35:36.119 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.115640+0000 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.120 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 1 Processor -- start 2026-02-24T18:35:36.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 1 -- start start 2026-02-24T18:35:36.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.114+0000 7fed8537e640 10 monclient: init 2026-02-24T18:35:36.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 Processor -- start 2026-02-24T18:35:36.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 -- start start 2026-02-24T18:35:36.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient: init 2026-02-24T18:35:36.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 Processor -- start 2026-02-24T18:35:36.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 -- start start 2026-02-24T18:35:36.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient: init 2026-02-24T18:35:36.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient: _add_conns ranks=[1,3,0,4,2,5] 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fed801527f0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fed80073530 0x7fed8015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-a con 0x7fed80073530 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fed8015f900 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed800739a0 con 0x7fed8015f900 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8015fd90 con 0x7fed801527f0 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fed80074b10 con 0x7fed80073530 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fed8537e640 10 monclient(hunting): authenticate will time out at 1479.796636s 2026-02-24T18:35:36.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient: _add_conns ranks=[2,1,5,4,3,0] 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe6241527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe624073530 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe62415fad0 0x7fe6240ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-c con 0x7fe62415fad0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6240739a0 con 0x7fe624073530 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62415d9d0 con 0x7fe6241527f0 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe624074b10 con 0x7fe62415fad0 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7fe62baea640 10 monclient(hunting): authenticate will time out at 1479.796756s 2026-02-24T18:35:36.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient: _add_conns ranks=[3,4,5,1,0,2] 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f295015f650 0x7f295015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): picked mon.noname-a con 0x7f295015f650 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f29500ceca0 0x7f29500d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): picked mon.noname-b con 0x7f29500ceca0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f29500d1ea0 0x7f29500c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): picked mon.noname-c con 0x7f29500d1ea0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f29500c8dd0 con 0x7f295015f650 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f29500d2450 con 0x7f29500ceca0 2026-02-24T18:35:36.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f29500d2730 con 0x7f29500d1ea0 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.115+0000 7f2955714640 10 monclient(hunting): authenticate will time out at 1479.796864s 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fed80073530 0x7fed8015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7f7fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fed8015f900 0x7fed800ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58678/0 (socket says 10.20.193.18:58678) 2026-02-24T18:35:36.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7f7fe640 1 -- 10.20.193.18:0/3005117293 learned_addr learned my addr 10.20.193.18:0/3005117293 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:36.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62a060640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe62415fad0 0x7fe6240ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62905e640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fe624073530 0x7fe62415f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51320/0 (socket says 10.20.193.18:51320) 2026-02-24T18:35:36.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62905e640 1 -- 10.20.193.18:0/2747797354 learned_addr learned my addr 10.20.193.18:0/2747797354 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:36.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): get_auth_request con 0x7fed80073530 auth_method 0 2026-02-24T18:35:36.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f29500d1ea0 0x7f29500c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62a060640 10 monclient(hunting): get_auth_request con 0x7fe62415fad0 auth_method 0 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62a060640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62a060640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fe62a060640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f29500d1ea0 0x7f29500c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44528/0 (socket says 10.20.193.18:44528) 2026-02-24T18:35:36.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 1 -- 10.20.193.18:0/590651848 learned_addr learned my addr 10.20.193.18:0/590651848 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 10 monclient(hunting): get_auth_request con 0x7f29500d1ea0 auth_method 0 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7f294f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.116+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.117+0000 7f294e7fc640 1 --2- 10.20.193.18:0/590651848 >> v2:10.20.193.32:3300/0 conn(0x7f29500ceca0 0x7f29500d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): get_auth_request con 0x7f29500ceca0 auth_method 0 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 10 monclient(hunting): handle_auth_done global_id 4514 payload 274 2026-02-24T18:35:36.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 1 -- 10.20.193.18:0/2747797354 >> v1:10.20.193.110:6789/0 conn(0x7fe6241527f0 legacy=0x7fe62415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 1 -- 10.20.193.18:0/2747797354 >> v1:10.20.193.32:6789/0 conn(0x7fe624073530 legacy=0x7fe62415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 1 monclient: found mon.noname-c 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fe62a060640 1 -- 10.20.193.18:0/2747797354 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe624073c20 con 0x7fe62415fad0 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_done global_id 4563 payload 274 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fed7e7fc640 1 -- 10.20.193.18:0/3005117293 >> v1:10.20.193.32:6789/0 conn(0x7fed801527f0 legacy=0x7fed8015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fed7e7fc640 1 -- 10.20.193.18:0/3005117293 >> v1:10.20.193.18:6789/0 conn(0x7fed8015f900 legacy=0x7fed800ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fed7e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.118+0000 7fed7e7fc640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7fed7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7fed7e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7fed7e7fc640 1 -- 10.20.193.18:0/3005117293 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8015d9d0 con 0x7fed80073530 2026-02-24T18:35:36.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 10 monclient(hunting): handle_auth_done global_id 4517 payload 274 2026-02-24T18:35:36.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 -- 10.20.193.18:0/590651848 >> v2:10.20.193.32:3300/0 conn(0x7f29500ceca0 msgr2=0x7f29500d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 --2- 10.20.193.18:0/590651848 >> v2:10.20.193.32:3300/0 conn(0x7f29500ceca0 0x7f29500d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 -- 10.20.193.18:0/590651848 >> v2:10.20.193.18:3300/0 conn(0x7f295015f650 msgr2=0x7f295015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 --2- 10.20.193.18:0/590651848 >> v2:10.20.193.18:3300/0 conn(0x7f295015f650 0x7f295015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.119+0000 7f294f7fe640 1 -- 10.20.193.18:0/590651848 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500c9020 con 0x7f29500d1ea0 2026-02-24T18:35:36.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.120+0000 7fed7e7fc640 1 --2- 10.20.193.18:0/3005117293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed8015f3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed6802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725eb917881a44da server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.120+0000 7fe62a060640 1 --2- 10.20.193.18:0/2747797354 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe62415fad0 0x7fe6240ccf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a38280d70c2593 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.121+0000 7f294f7fe640 1 --2- 10.20.193.18:0/590651848 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f29500c9b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29500ce0a0 tx=0x7f294002f200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21414e0707cec86f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 1 -- 10.20.193.18:0/3005117293 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6803c070 con 0x7fed80073530 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: dump: 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.143 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 1 -- 10.20.193.18:0/3005117293 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6802fb70 con 0x7fed80073530 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 5 monclient: authenticate success, global_id 4563 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 1 -- 10.20.193.18:0/3005117293 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68037440 con 0x7fed80073530 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 4 set_mon_vals no callback set 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed7dffb640 10 monclient: dump: 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 10 monclient: shutdown 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 -- 10.20.193.18:0/3005117293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 msgr2=0x7fed8015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 --2- 10.20.193.18:0/3005117293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed8015f3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed6802ee60 comp rx=0 tx=0).stop 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 -- 10.20.193.18:0/3005117293 shutdown_connections 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 --2- 10.20.193.18:0/3005117293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed8015f3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 -- 10.20.193.18:0/3005117293 >> 10.20.193.18:0/3005117293 conn(0x7fed800ce730 msgr2=0x7fed800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 -- 10.20.193.18:0/3005117293 shutdown_connections 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.123+0000 7fed8537e640 1 -- 10.20.193.18:0/3005117293 wait complete. 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient: monmap: 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.146 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.147 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.147 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 1 -- 10.20.193.18:0/590651848 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294002fc70 con 0x7f29500d1ea0 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: dump: 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.148 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 1 -- 10.20.193.18:0/590651848 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294002fe10 con 0x7f29500d1ea0 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 1 -- 10.20.193.18:0/590651848 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940037610 con 0x7f29500d1ea0 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f294dffb640 10 monclient: dump: 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 5 monclient: authenticate success, global_id 4517 2026-02-24T18:35:36.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 4 set_mon_vals no callback set 2026-02-24T18:35:36.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 librados: starting msgr at 2026-02-24T18:35:36.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 librados: starting objecter 2026-02-24T18:35:36.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 Processor -- start 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 1 -- 10.20.193.18:0/2747797354 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62003c070 con 0x7fe62415fad0 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: dump: 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.152 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 1 -- 10.20.193.18:0/2747797354 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe620037440 con 0x7fe62415fad0 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 1 -- 10.20.193.18:0/2747797354 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620037810 con 0x7fe62415fad0 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 5 monclient: authenticate success, global_id 4514 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 4 set_mon_vals no callback set 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62885d640 10 monclient: dump: 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.153 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 -- start start 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 librados: setting wanted keys 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 librados: calling monclient init 2026-02-24T18:35:36.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient: init 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 10 monclient: shutdown 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 -- 10.20.193.18:0/590651848 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 msgr2=0x7f29500c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 --2- 10.20.193.18:0/590651848 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f29500c9b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29500ce0a0 tx=0x7f294002f200 comp rx=0 tx=0).stop 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 10 monclient: shutdown 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 -- 10.20.193.18:0/2747797354 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe62415fad0 msgr2=0x7fe6240ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 --2- 10.20.193.18:0/2747797354 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe62415fad0 0x7fe6240ccf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002f230 comp rx=0 tx=0).stop 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 -- 10.20.193.18:0/590651848 shutdown_connections 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 --2- 10.20.193.18:0/590651848 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f29500c9b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 --2- 10.20.193.18:0/590651848 >> v2:10.20.193.32:3300/0 conn(0x7f29500ceca0 0x7f29500d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 --2- 10.20.193.18:0/590651848 >> v2:10.20.193.18:3300/0 conn(0x7f295015f650 0x7f295015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 -- 10.20.193.18:0/590651848 >> 10.20.193.18:0/590651848 conn(0x7f2950072fc0 msgr2=0x7f29500733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 -- 10.20.193.18:0/590651848 shutdown_connections 2026-02-24T18:35:36.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7f2955714640 1 -- 10.20.193.18:0/590651848 wait complete. 2026-02-24T18:35:36.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 -- 10.20.193.18:0/2747797354 shutdown_connections 2026-02-24T18:35:36.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 --2- 10.20.193.18:0/2747797354 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe62415fad0 0x7fe6240ccf70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 -- 10.20.193.18:0/2747797354 >> 10.20.193.18:0/2747797354 conn(0x7fe6240ce730 msgr2=0x7fe6240ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 -- 10.20.193.18:0/2747797354 shutdown_connections 2026-02-24T18:35:36.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fe62baea640 1 -- 10.20.193.18:0/2747797354 wait complete. 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-a con 0x7fed80073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.124+0000 7fed8537e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fed801527f0 0x7fed800ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-c con 0x7fed801527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fed8015f900 0x7fed800ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): picked mon.noname-b con 0x7fed8015f900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fed800741e0 con 0x7fed80073530 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed80073f00 con 0x7fed8015f900 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed800744c0 con 0x7fed801527f0 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed8537e640 10 monclient(hunting): authenticate will time out at 1479.806025s 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 10 monclient: monmap: 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.159 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 10 monclient: monmap: 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.160 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41820/0 (socket says 10.20.193.18:41820) 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 1 -- 10.20.193.18:0/738449203 learned_addr learned my addr 10.20.193.18:0/738449203 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): get_auth_request con 0x7fed80073530 auth_method 0 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 librados: starting msgr at 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 librados: starting objecter 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 librados: starting msgr at 2026-02-24T18:35:36.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 librados: starting objecter 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7f7fe640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fed8015f900 0x7fed800ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 Processor -- start 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 Processor -- start 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 -- start start 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 librados: setting wanted keys 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 1 librados: calling monclient init 2026-02-24T18:35:36.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fe62baea640 10 monclient: init 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 -- start start 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 librados: setting wanted keys 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 1 librados: calling monclient init 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7f2955714640 10 monclient: init 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7f7fe640 10 monclient(hunting): get_auth_request con 0x7fed8015f900 auth_method 0 2026-02-24T18:35:36.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.125+0000 7fed7e7fc640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fed801527f0 0x7fed800ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): get_auth_request con 0x7fed801527f0 auth_method 0 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe624073530 0x7fe6241f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-a con 0x7fe624073530 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-c con 0x7fe6241527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe62415fad0 0x7fe6241f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): picked mon.noname-b con 0x7fe62415fad0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6240744c0 con 0x7fe624073530 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6240741e0 con 0x7fe62415fad0 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6240747a0 con 0x7fe6241527f0 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62baea640 10 monclient(hunting): authenticate will time out at 1479.807088s 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f29500ceca0 0x7f29501e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): picked mon.noname-b con 0x7f29500ceca0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): picked mon.noname-c con 0x7f29500d1ea0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f295015f650 0x7f2950157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): picked mon.noname-a con 0x7f295015f650 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f29500cf0f0 con 0x7f295015f650 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29500c95e0 con 0x7f29500ceca0 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ea370 con 0x7f29500d1ea0 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f2955714640 10 monclient(hunting): authenticate will time out at 1479.807117s 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fed7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7fe62905e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.126+0000 7f294e7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 10 monclient(hunting): handle_auth_done global_id 4569 payload 824 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 -- 10.20.193.18:0/738449203 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fed801527f0 msgr2=0x7fed800ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fed801527f0 0x7fed800ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 -- 10.20.193.18:0/738449203 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fed8015f900 msgr2=0x7fed800ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fed8015f900 0x7fed800ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.127+0000 7fed7effd640 1 -- 10.20.193.18:0/738449203 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed800c9070 con 0x7fed80073530 2026-02-24T18:35:36.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44560/0 (socket says 10.20.193.18:44560) 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 1 -- 10.20.193.18:0/3664879540 learned_addr learned my addr 10.20.193.18:0/3664879540 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): get_auth_request con 0x7fe6241527f0 auth_method 0 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fed7effd640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c5b0 tx=0x7fed7400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fed5f7fe640 10 client.?.objecter ms_handle_connect 0x7fed80073530 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fed5f7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44562/0 (socket says 10.20.193.18:44562) 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 1 -- 10.20.193.18:0/718671042 learned_addr learned my addr 10.20.193.18:0/718671042 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 10 monclient(hunting): get_auth_request con 0x7f29500d1ea0 auth_method 0 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7f294e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.128+0000 7fe62905e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7fe62a060640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe62415fad0 0x7fe6241f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7f294effd640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f29500ceca0 0x7f29501e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7fe62a060640 10 monclient(hunting): get_auth_request con 0x7fe62415fad0 auth_method 0 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7fe62a060640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7fe62a060640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.129+0000 7fe62a060640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62985f640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe624073530 0x7fe6241f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294effd640 10 monclient(hunting): get_auth_request con 0x7f29500ceca0 auth_method 0 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62985f640 10 monclient(hunting): get_auth_request con 0x7fe624073530 auth_method 0 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62985f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62985f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62985f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62a060640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 10 monclient(hunting): handle_auth_done global_id 4523 payload 824 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 -- 10.20.193.18:0/3664879540 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe62415fad0 msgr2=0x7fe6241f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe62415fad0 0x7fe6241f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 -- 10.20.193.18:0/3664879540 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe624073530 msgr2=0x7fe6241f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe624073530 0x7fe6241f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 monclient: found mon.noname-c 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fe62905e640 1 -- 10.20.193.18:0/3664879540 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6240c70a0 con 0x7fe6241527f0 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740052d0 con 0x7fed80073530 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: dump: 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed74005470 con 0x7fed80073530 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740038f0 con 0x7fed80073530 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 5 monclient: authenticate success, global_id 4569 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 10 monclient: _renew_subs 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed7d7fa640 4 set_mon_vals no callback set 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed800c9350 con 0x7fed80073530 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed5f7fe640 10 monclient: dump: 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 10 client.4569.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 10 monclient: _renew_subs 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed800c9910 con 0x7fed80073530 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7fed8537e640 1 librados: init done 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.130+0000 7f294f7fe640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f295015f650 0x7f2950157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7fed8537e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7fed8537e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7fed8537e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed800747a0 con 0x7fed80073530 2026-02-24T18:35:36.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): get_auth_request con 0x7f295015f650 auth_method 0 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.131+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f0b4ccea640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f0b4ccea640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f0b4ccea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0b48153f20 con 0x7f0b480ccb30 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 10 monclient(hunting): handle_auth_done global_id 4526 payload 824 2026-02-24T18:35:36.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 -- 10.20.193.18:0/718671042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f29500ceca0 msgr2=0x7f29501e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f29500ceca0 0x7f29501e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 -- 10.20.193.18:0/718671042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f295015f650 msgr2=0x7f2950157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f295015f650 0x7f2950157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 monclient: found mon.noname-c 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.132+0000 7f294e7fc640 1 -- 10.20.193.18:0/718671042 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29501ea650 con 0x7f29500d1ea0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7fe62905e640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c730 tx=0x7fe61800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7fe6127fc640 10 client.?.objecter ms_handle_connect 0x7fe6241527f0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7fe6127fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7f294e7fc640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f293800e890 tx=0x7f293800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7f292f7fe640 10 client.?.objecter ms_handle_connect 0x7f29500d1ea0 2026-02-24T18:35:36.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.133+0000 7f292f7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618017070 con 0x7fe6241527f0 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: dump: 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618005b90 con 0x7fe6241527f0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 5 monclient: authenticate success, global_id 4523 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 10 monclient: _renew_subs 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6240c7340 con 0x7fe6241527f0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618007070 con 0x7fe6241527f0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe613fff640 4 set_mon_vals no callback set 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe6127fc640 10 monclient: dump: 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 10 client.4523.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 10 monclient: _renew_subs 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6240c7900 con 0x7fe6241527f0 2026-02-24T18:35:36.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.138+0000 7fe62baea640 1 librados: init done 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938017070 con 0x7f29500d1ea0 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: dump: 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293800c940 con 0x7f29500d1ea0 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 5 monclient: authenticate success, global_id 4526 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _renew_subs 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501ea930 con 0x7f29500d1ea0 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293800cd30 con 0x7f29500d1ea0 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f294d7fa640 4 set_mon_vals no callback set 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f292f7fe640 10 monclient: dump: 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 client.4526.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _renew_subs 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501eaef0 con 0x7f29500d1ea0 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 1 librados: init done 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7fe62baea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7fe62baea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7fe62baea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe624074fa0 con 0x7fe6241527f0 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.139+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29500cad70 con 0x7f29500d1ea0 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.143+0000 7f0b2effd640 1 -- 10.20.193.18:0/555065449 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0b3c061dc0 con 0x7f0b480ccb30 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.143+0000 7f0b2effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.143+0000 7f0b2effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.182 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 10 librados: watch_flush enter 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 10 librados: watch_flush exit 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 20 client.4511.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 10 client.4511.objecter successfully canceled tick 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 msgr2=0x7f0b1407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 0x7f0b1407b2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0b48069a40 tx=0x7f0b30031000 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 10 monclient: shutdown 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 msgr2=0x7f0b481e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b500 tx=0x7f0b3c00b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 shutdown_connections 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0b14078e00 0x7f0b1407b2c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0b48156050 0x7f0b480c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0b48155c70 0x7f0b480c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 --2- 10.20.193.18:0/555065449 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0b480ccb30 0x7f0b481e9960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 >> 10.20.193.18:0/555065449 conn(0x7f0b480cc010 msgr2=0x7f0b480cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 shutdown_connections 2026-02-24T18:35:36.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.145+0000 7f0b4ccea640 1 -- 10.20.193.18:0/555065449 wait complete. 2026-02-24T18:35:36.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.146+0000 7f0b4ccea640 1 librados: shutdown 2026-02-24T18:35:36.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.4 2026-02-24T18:35:36.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.240+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fed74003a90 con 0x7fed80073530 2026-02-24T18:35:36.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.241+0000 7fed5f7fe640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 0x7fed4c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.241+0000 7fed7e7fc640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 0x7fed4c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.241+0000 7fed7e7fc640 10 monclient: get_auth_request con 0x7fed4c078e50 auth_method 0 2026-02-24T18:35:36.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.241+0000 7fed5f7fe640 10 client.4569.objecter ms_handle_connect 0x7fed4c078e50 2026-02-24T18:35:36.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.241+0000 7fed7e7fc640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 0x7fed4c07b310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed68039000 tx=0x7fed68033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.248+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fed7405e020 con 0x7fed80073530 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.248+0000 7fed5f7fe640 10 client.4569.objecter ms_dispatch2 0x7fed80000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.248+0000 7fed5f7fe640 3 client.4569.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.248+0000 7fed5f7fe640 3 client.4569.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.249+0000 7fed5f7fe640 20 client.4569.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.249+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fed8015fd90 con 0x7fed80073530 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.249+0000 7fed5f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.249+0000 7fed5f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c803c040 con 0x7f19dc072e50 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 10 monclient: dump: 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.326 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 5 monclient: authenticate success, global_id 4513 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 10 monclient: _renew_subs 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc157de0 con 0x7f19dc072e50 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 10 client.4513.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 10 monclient: _renew_subs 2026-02-24T18:35:36.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc1583a0 con 0x7f19dc072e50 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.324+0000 7f19e1217640 1 librados: init done 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.325+0000 7f19e1217640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.325+0000 7f19e1217640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.325+0000 7f19e1217640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.325+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc0732c0 con 0x7f19dc072e50 2026-02-24T18:35:36.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.333+0000 7fed8537e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:36.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.333+0000 7fed8537e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:36.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.333+0000 7fed8537e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.333+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7fed800cf670 con 0x7fed80073530 2026-02-24T18:35:36.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed5f7fe640 1 -- 10.20.193.18:0/738449203 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fed74061fe0 con 0x7fed80073530 2026-02-24T18:35:36.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed5f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:36.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed5f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.338 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:36.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe618007210 con 0x7fe6241527f0 2026-02-24T18:35:36.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fe6127fc640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 0x7fe5f407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed8537e640 10 librados: watch_flush enter 2026-02-24T18:35:36.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed8537e640 10 librados: watch_flush exit 2026-02-24T18:35:36.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.336+0000 7fed8537e640 20 client.4569.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 10 client.4569.objecter successfully canceled tick 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 msgr2=0x7fed4c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 0x7fed4c07b310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed68039000 tx=0x7fed68033000 comp rx=0 tx=0).stop 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 10 monclient: shutdown 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 msgr2=0x7fed801f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c5b0 tx=0x7fed7400ca80 comp rx=0 tx=0).stop 2026-02-24T18:35:36.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fe62985f640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 0x7fe5f407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fe62985f640 10 monclient: get_auth_request con 0x7fe5f4078e00 auth_method 0 2026-02-24T18:35:36.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 shutdown_connections 2026-02-24T18:35:36.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fed4c078e50 0x7fed4c07b310 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fed8015f900 0x7fed800ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fed801527f0 0x7fed800ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 --2- 10.20.193.18:0/738449203 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fed80073530 0x7fed801f7210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 >> 10.20.193.18:0/738449203 conn(0x7fed800ce730 msgr2=0x7fed8015ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 shutdown_connections 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 -- 10.20.193.18:0/738449203 wait complete. 2026-02-24T18:35:36.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fe62985f640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 0x7fe5f407b2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe614002770 tx=0x7fe614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fe6127fc640 10 client.4523.objecter ms_handle_connect 0x7fe5f4078e00 2026-02-24T18:35:36.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.337+0000 7fed8537e640 1 librados: shutdown 2026-02-24T18:35:36.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.338+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fe61805d4a0 con 0x7fe6241527f0 2026-02-24T18:35:36.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.338+0000 7fe6127fc640 10 client.4523.objecter ms_dispatch2 0x7fe624000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.338+0000 7fe6127fc640 3 client.4523.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.338+0000 7fe6127fc640 3 client.4523.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.338+0000 7fe6127fc640 20 client.4523.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.345+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe618021020 con 0x7fe6241527f0 2026-02-24T18:35:36.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.345+0000 7fe6127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.345+0000 7fe6127fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.12 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c802fad0 con 0x7f19dc072e50 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c803f880 con 0x7f19dc072e50 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19d8ff9640 10 monclient: dump: 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.350+0000 7f19da7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:36.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.351+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f19c803fa20 con 0x7f19dc072e50 2026-02-24T18:35:36.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.351+0000 7f19d8ff9640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 0x7f19ac07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.352+0000 7f19db7fe640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 0x7f19ac07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.352+0000 7f19db7fe640 10 monclient: get_auth_request con 0x7f19ac078e00 auth_method 0 2026-02-24T18:35:36.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.352+0000 7f19db7fe640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 0x7f19ac07b2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19dc06f1b0 tx=0x7f19cc006cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.352+0000 7f19d8ff9640 10 client.4513.objecter ms_handle_connect 0x7f19ac078e00 2026-02-24T18:35:36.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.353+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2938013470 con 0x7f29500d1ea0 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.353+0000 7f292f7fe640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 0x7f291c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.354+0000 7f294effd640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 0x7f291c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.354+0000 7f294effd640 10 monclient: get_auth_request con 0x7f291c078e80 auth_method 0 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.354+0000 7f292f7fe640 10 client.4526.objecter ms_handle_connect 0x7f291c078e80 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.354+0000 7f294effd640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 0x7f291c07b340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2944006fd0 tx=0x7f2944009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f15d97fa640 1 -- 10.20.193.18:0/2072914565 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f15cc062720 con 0x7f15e41527f0 2026-02-24T18:35:36.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f15d97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:36.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f15d97fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.357 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:36.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f293805e350 con 0x7f29500d1ea0 2026-02-24T18:35:36.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f292f7fe640 10 client.4526.objecter ms_dispatch2 0x7f2950000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f292f7fe640 3 client.4526.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f292f7fe640 3 client.4526.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.355+0000 7f292f7fe640 20 client.4526.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 1 -- 10.20.193.18:0/585681122 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e003c070 con 0x7fa0ec074d60 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: dump: 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.360 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 1 -- 10.20.193.18:0/585681122 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e002fb40 con 0x7fa0ec074d60 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 1 -- 10.20.193.18:0/585681122 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0037440 con 0x7fa0ec074d60 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 5 monclient: authenticate success, global_id 4519 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 4 set_mon_vals no callback set 2026-02-24T18:35:36.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0dbfff640 10 monclient: dump: 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 10 monclient: shutdown 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 -- 10.20.193.18:0/585681122 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec074d60 msgr2=0x7fa0ec0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 --2- 10.20.193.18:0/585681122 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0cc210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec08f620 tx=0x7fa0e002eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 -- 10.20.193.18:0/585681122 shutdown_connections 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 --2- 10.20.193.18:0/585681122 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0cc210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 --2- 10.20.193.18:0/585681122 >> v2:10.20.193.18:3300/0 conn(0x7fa0ec155e70 0x7fa0ec074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 -- 10.20.193.18:0/585681122 >> 10.20.193.18:0/585681122 conn(0x7fa0ec0ce700 msgr2=0x7fa0ec0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 -- 10.20.193.18:0/585681122 shutdown_connections 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.356+0000 7fa0f3914640 1 -- 10.20.193.18:0/585681122 wait complete. 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient: monmap: 2026-02-24T18:35:36.363 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 librados: starting msgr at 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 librados: starting objecter 2026-02-24T18:35:36.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 Processor -- start 2026-02-24T18:35:36.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 10 librados: watch_flush enter 2026-02-24T18:35:36.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 10 librados: watch_flush exit 2026-02-24T18:35:36.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 -- start start 2026-02-24T18:35:36.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 librados: setting wanted keys 2026-02-24T18:35:36.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 librados: calling monclient init 2026-02-24T18:35:36.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient: init 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 20 client.4498.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 10 client.4498.objecter successfully canceled tick 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 msgr2=0x7f15b407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 0x7f15b407b2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15d4007fd0 tx=0x7f15d4007b40 comp rx=0 tx=0).stop 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 10 monclient: shutdown 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 msgr2=0x7f15e41f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b570 tx=0x7f15cc00ba40 comp rx=0 tx=0).stop 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 shutdown_connections 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15b4078e30 0x7f15b407b2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f15e41527f0 0x7f15e41f4b00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f15e415fad0 0x7f15e40c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 --2- 10.20.193.18:0/2072914565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f15e4073530 0x7f15e40c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 >> 10.20.193.18:0/2072914565 conn(0x7f15e40ce730 msgr2=0x7f15e40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 shutdown_connections 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 -- 10.20.193.18:0/2072914565 wait complete. 2026-02-24T18:35:36.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7f15eaaea640 1 librados: shutdown 2026-02-24T18:35:36.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-a con 0x7fa0ec074d60 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec155e70 0x7fa0ec147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-b con 0x7fa0ec155e70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0ec156250 0x7fa0ec141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): picked mon.noname-c con 0x7fa0ec156250 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec074660 con 0x7fa0ec074d60 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cd9b0 con 0x7fa0ec155e70 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cc750 con 0x7fa0ec156250 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.357+0000 7fa0f3914640 10 monclient(hunting): authenticate will time out at 1480.038839s 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41844/0 (socket says 10.20.193.18:41844) 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 1 -- 10.20.193.18:0/2917843233 learned_addr learned my addr 10.20.193.18:0/2917843233 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): get_auth_request con 0x7fa0ec074d60 auth_method 0 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.358+0000 7fa0f1689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.359+0000 7fa0f1689640 10 monclient(hunting): handle_auth_done global_id 4578 payload 824 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 -- 10.20.193.18:0/2917843233 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0ec156250 msgr2=0x7fa0ec141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0ec156250 0x7fa0ec141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 -- 10.20.193.18:0/2917843233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec155e70 msgr2=0x7fa0ec147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec155e70 0x7fa0ec147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.360+0000 7fa0f1689640 1 -- 10.20.193.18:0/2917843233 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0c5980 con 0x7fa0ec074d60 2026-02-24T18:35:36.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.361+0000 7fa0f1689640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b500 tx=0x7fa0dc00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.361+0000 7fa0d9ffb640 10 client.?.objecter ms_handle_connect 0x7fa0ec074d60 2026-02-24T18:35:36.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.361+0000 7fa0d9ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.362+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2938062310 con 0x7f29500d1ea0 2026-02-24T18:35:36.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.362+0000 7f292f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.362+0000 7f292f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc004480 con 0x7fa0ec074d60 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: dump: 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.376 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 5 monclient: authenticate success, global_id 4578 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0dc014070 con 0x7fa0ec074d60 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 10 monclient: _renew_subs 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec0c5c60 con 0x7fa0ec074d60 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc002920 con 0x7fa0ec074d60 2026-02-24T18:35:36.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0db7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0d9ffb640 10 monclient: dump: 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.378 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 10 client.4578.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 10 monclient: _renew_subs 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec0c6220 con 0x7fa0ec074d60 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.363+0000 7fa0f3914640 1 librados: init done 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.364+0000 7fa0f3914640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.364+0000 7fa0f3914640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.364+0000 7fa0f3914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.364+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec069800 con 0x7fa0ec074d60 2026-02-24T18:35:36.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.15 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.376+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa0dc002ac0 con 0x7fa0ec074d60 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.376+0000 7fa0d9ffb640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 0x7fa0bc07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.377+0000 7fa0f0e88640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 0x7fa0bc07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.377+0000 7fa0f0e88640 10 monclient: get_auth_request con 0x7fa0bc078e30 auth_method 0 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.377+0000 7fa0f0e88640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 0x7fa0bc07b2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec08f080 tx=0x7fa0e0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.377+0000 7fa0d9ffb640 10 client.4578.objecter ms_handle_connect 0x7fa0bc078e30 2026-02-24T18:35:36.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.429+0000 7fe62baea640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:36.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.429+0000 7fe62baea640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:36.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.429+0000 7fe62baea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.429+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe6241503f0 con 0x7fe6241527f0 2026-02-24T18:35:36.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.432+0000 7fe6127fc640 1 -- 10.20.193.18:0/3664879540 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6240739a0 con 0x7fe6241527f0 2026-02-24T18:35:36.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.432+0000 7fe6127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:36.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.432+0000 7fe6127fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.434 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:36.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 10 librados: watch_flush enter 2026-02-24T18:35:36.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 10 librados: watch_flush exit 2026-02-24T18:35:36.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 20 client.4523.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 10 client.4523.objecter successfully canceled tick 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 msgr2=0x7fe5f407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 0x7fe5f407b2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe614002770 tx=0x7fe614009040 comp rx=0 tx=0).stop 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 10 monclient: shutdown 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 msgr2=0x7fe6240c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c730 tx=0x7fe61800cc00 comp rx=0 tx=0).stop 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 shutdown_connections 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe5f4078e00 0x7fe5f407b2c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe62415fad0 0x7fe6241f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe6241527f0 0x7fe6240c8100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 --2- 10.20.193.18:0/3664879540 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe624073530 0x7fe6241f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 >> 10.20.193.18:0/3664879540 conn(0x7fe6240ce730 msgr2=0x7fe62406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 shutdown_connections 2026-02-24T18:35:36.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 -- 10.20.193.18:0/3664879540 wait complete. 2026-02-24T18:35:36.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.434+0000 7fe62baea640 1 librados: shutdown 2026-02-24T18:35:36.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.1 2026-02-24T18:35:36.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.448+0000 7f2955714640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:36.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.448+0000 7f2955714640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:36.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.448+0000 7f2955714640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.448+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f295015d7b0 con 0x7f29500d1ea0 2026-02-24T18:35:36.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.450+0000 7f292f7fe640 1 -- 10.20.193.18:0/718671042 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2938066610 con 0x7f29500d1ea0 2026-02-24T18:35:36.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.450+0000 7f292f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:36.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.450+0000 7f292f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.452 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 10 librados: watch_flush enter 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 10 librados: watch_flush exit 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 20 client.4526.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 10 client.4526.objecter successfully canceled tick 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 msgr2=0x7f291c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 0x7f291c07b340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2944006fd0 tx=0x7f2944009190 comp rx=0 tx=0).stop 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 10 monclient: shutdown 2026-02-24T18:35:36.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 msgr2=0x7f2950157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f293800e890 tx=0x7f293800ed60 comp rx=0 tx=0).stop 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 shutdown_connections 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f291c078e80 0x7f291c07b340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f295015f650 0x7f2950157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f29500d1ea0 0x7f2950157aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 --2- 10.20.193.18:0/718671042 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f29500ceca0 0x7f29501e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 >> 10.20.193.18:0/718671042 conn(0x7f2950072fc0 msgr2=0x7f29500cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 shutdown_connections 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 -- 10.20.193.18:0/718671042 wait complete. 2026-02-24T18:35:36.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.451+0000 7f2955714640 1 librados: shutdown 2026-02-24T18:35:36.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 1 -- 10.20.193.18:0/4206584613 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec03c070 con 0x7f06fc06ff20 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: dump: 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.459 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 1 -- 10.20.193.18:0/4206584613 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ec02fae0 con 0x7f06fc06ff20 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 1 -- 10.20.193.18:0/4206584613 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec037440 con 0x7f06fc06ff20 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 5 monclient: authenticate success, global_id 4525 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 4 set_mon_vals no callback set 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0700a09640 10 monclient: dump: 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.461 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 10 monclient: shutdown 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 -- 10.20.193.18:0/4206584613 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc06ff20 msgr2=0x7f06fc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 --2- 10.20.193.18:0/4206584613 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc06ff20 0x7f06fc070300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0098a0 tx=0x7f06ec02ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 -- 10.20.193.18:0/4206584613 shutdown_connections 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 --2- 10.20.193.18:0/4206584613 >> v2:10.20.193.18:3300/0 conn(0x7f06fc15a5e0 0x7f06fc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 --2- 10.20.193.18:0/4206584613 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc06ff20 0x7f06fc070300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 -- 10.20.193.18:0/4206584613 >> 10.20.193.18:0/4206584613 conn(0x7f06fc0cd780 msgr2=0x7f06fc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 -- 10.20.193.18:0/4206584613 shutdown_connections 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.454+0000 7f0703c96640 1 -- 10.20.193.18:0/4206584613 wait complete. 2026-02-24T18:35:36.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 10 monclient: monmap: 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 librados: starting msgr at 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 librados: starting objecter 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 Processor -- start 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 -- start start 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 librados: setting wanted keys 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 1 librados: calling monclient init 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.455+0000 7f0703c96640 10 monclient: init 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f06fc06ff20 0x7f06fc06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-a con 0x7f06fc06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-c con 0x7f06fc0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc15a5e0 0x7f06fc0bba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): picked mon.noname-b con 0x7f06fc15a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc150350 con 0x7f06fc06ff20 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc0c8a10 con 0x7f06fc15a5e0 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1504d0 con 0x7f06fc0c99a0 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f0703c96640 10 monclient(hunting): authenticate will time out at 1480.137228s 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44576/0 (socket says 10.20.193.18:44576) 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 1 -- 10.20.193.18:0/1625320873 learned_addr learned my addr 10.20.193.18:0/1625320873 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 10 monclient(hunting): get_auth_request con 0x7f06fc0c99a0 auth_method 0 2026-02-24T18:35:36.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f070120a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc046020 con 0x7f7d0015d730 2026-02-24T18:35:36.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: dump: 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc041450 con 0x7f7d0015d730 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc037e60 con 0x7f7d0015d730 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7d073ce640 5 monclient: authenticate success, global_id 4516 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7d073ce640 10 monclient: _renew_subs 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7d073ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d000c7410 con 0x7f7d0015d730 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cef7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.456+0000 7f7cedffb640 10 monclient: dump: 2026-02-24T18:35:36.469 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 client.4516.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 monclient: _renew_subs 2026-02-24T18:35:36.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d000c79d0 con 0x7f7d0015d730 2026-02-24T18:35:36.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 1 librados: init done 2026-02-24T18:35:36.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f070120a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f070120a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f070120a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f19c803fd40 con 0x7f19dc072e50 2026-02-24T18:35:36.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f19d8ff9640 10 client.4513.objecter ms_dispatch2 0x7f19dc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d00152bd0 con 0x7f7d0015d730 2026-02-24T18:35:36.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f19d8ff9640 3 client.4513.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f19d8ff9640 3 client.4513.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.457+0000 7f19d8ff9640 20 client.4513.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 10 monclient(hunting): handle_auth_done global_id 4529 payload 824 2026-02-24T18:35:36.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 -- 10.20.193.18:0/1625320873 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc15a5e0 msgr2=0x7f06fc0bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc15a5e0 0x7f06fc0bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 -- 10.20.193.18:0/1625320873 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f06fc06ff20 msgr2=0x7f06fc06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f06fc06ff20 0x7f06fc06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 monclient: found mon.noname-c 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.458+0000 7f070120a640 1 -- 10.20.193.18:0/1625320873 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc0ba4a0 con 0x7f06fc0c99a0 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.459+0000 7f070120a640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c760 tx=0x7f06f000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.459+0000 7f06ea7fc640 10 client.?.objecter ms_handle_connect 0x7f06fc0c99a0 2026-02-24T18:35:36.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.459+0000 7f06ea7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.9 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0017070 con 0x7f06fc0c99a0 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: dump: 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.478 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f0005b90 con 0x7f06fc0c99a0 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0007070 con 0x7f06fc0c99a0 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 5 monclient: authenticate success, global_id 4529 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ebfff640 4 set_mon_vals no callback set 2026-02-24T18:35:36.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 10 monclient: _renew_subs 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc0ba780 con 0x7f06fc0c99a0 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f06ea7fc640 10 monclient: dump: 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.480 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 10 client.4529.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 10 monclient: _renew_subs 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc0bad40 con 0x7f06fc0c99a0 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.461+0000 7f0703c96640 1 librados: init done 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.462+0000 7f0703c96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.462+0000 7f0703c96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.462+0000 7f0703c96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.462+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc150860 con 0x7f06fc0c99a0 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.466+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f19c8088120 con 0x7f19dc072e50 2026-02-24T18:35:36.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.466+0000 7f19d8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.466+0000 7f19d8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.473+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f06f0007210 con 0x7f06fc0c99a0 2026-02-24T18:35:36.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.473+0000 7f06ea7fc640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 0x7f06cc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.474+0000 7f0701a0b640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 0x7f06cc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.474+0000 7f0701a0b640 10 monclient: get_auth_request con 0x7f06cc078e00 auth_method 0 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.474+0000 7f06ea7fc640 10 client.4529.objecter ms_handle_connect 0x7f06cc078e00 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.474+0000 7f0701a0b640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 0x7f06cc07b2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06ec039000 tx=0x7f06ec031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.475+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f06f005d4b0 con 0x7f06fc0c99a0 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.475+0000 7f06ea7fc640 10 client.4529.objecter ms_dispatch2 0x7f06fc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.475+0000 7f06ea7fc640 3 client.4529.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.475+0000 7f06ea7fc640 3 client.4529.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.475+0000 7f06ea7fc640 20 client.4529.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa0dc05de40 con 0x7fa0ec074d60 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 10 client.4578.objecter ms_dispatch2 0x7fa0ec000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 3 client.4578.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 3 client.4578.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 20 client.4578.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa0dc062500 con 0x7fa0ec074d60 2026-02-24T18:35:36.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.485+0000 7fa0d9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.551+0000 7f19e1217640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:36.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.551+0000 7f19e1217640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:36.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.551+0000 7f19e1217640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.551+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f19dc150cf0 con 0x7f19dc072e50 2026-02-24T18:35:36.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.568+0000 7fa0f3914640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:36.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.568+0000 7fa0f3914640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:36.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.568+0000 7fa0f3914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.568+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa0ec15e8a0 con 0x7fa0ec074d60 2026-02-24T18:35:36.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.570+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7cfc041840 con 0x7f7d0015d730 2026-02-24T18:35:36.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.570+0000 7f7cedffb640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 0x7f7cd007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7d05143640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 0x7f7cd007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7d05143640 10 monclient: get_auth_request con 0x7f7cd0078e00 auth_method 0 2026-02-24T18:35:36.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7fa0d9ffb640 1 -- 10.20.193.18:0/2917843233 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0dc061e00 con 0x7fa0ec074d60 2026-02-24T18:35:36.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7fa0d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:36.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7fa0d9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f7cfc083a90 con 0x7f7d0015d730 2026-02-24T18:35:36.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 10 client.4516.objecter ms_dispatch2 0x7f7d00000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 3 client.4516.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 3 client.4516.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.574 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:36.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 20 client.4516.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7cedffb640 10 client.4516.objecter ms_handle_connect 0x7f7cd0078e00 2026-02-24T18:35:36.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.571+0000 7f7d05143640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 0x7f7cd007b2c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098a0 tx=0x7f7cf0006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 10 librados: watch_flush enter 2026-02-24T18:35:36.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 10 librados: watch_flush exit 2026-02-24T18:35:36.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 20 client.4578.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 10 client.4578.objecter successfully canceled tick 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 msgr2=0x7fa0bc07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 0x7fa0bc07b2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec08f080 tx=0x7fa0e0031000 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 10 monclient: shutdown 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 msgr2=0x7fa0ec0c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b500 tx=0x7fa0dc00b9d0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 shutdown_connections 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa0bc078e30 0x7fa0bc07b2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa0ec155e70 0x7fa0ec147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa0ec156250 0x7fa0ec141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 --2- 10.20.193.18:0/2917843233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa0ec074d60 0x7fa0ec0c6a20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 >> 10.20.193.18:0/2917843233 conn(0x7fa0ec0ce700 msgr2=0x7fa0ec15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 shutdown_connections 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 -- 10.20.193.18:0/2917843233 wait complete. 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.572+0000 7fa0f3914640 1 librados: shutdown 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.575+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f06f0021020 con 0x7f06fc0c99a0 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.576+0000 7f06ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.576+0000 7f06ea7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.577+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7cfc050030 con 0x7f7d0015d730 2026-02-24T18:35:36.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.577+0000 7f7cedffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.577+0000 7f7cedffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.8 2026-02-24T18:35:36.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.585+0000 7f19d8ff9640 1 -- 10.20.193.18:0/3124514010 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19c8087a20 con 0x7f19dc072e50 2026-02-24T18:35:36.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.585+0000 7f19d8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:36.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.585+0000 7f19d8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.587 INFO:teuthology.orchestra.run.trial018.stdout:90194313219 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 10 librados: watch_flush enter 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 10 librados: watch_flush exit 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 20 client.4513.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 10 client.4513.objecter successfully canceled tick 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 msgr2=0x7f19ac07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 0x7f19ac07b2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19dc06f1b0 tx=0x7f19cc006cf0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 10 monclient: shutdown 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 msgr2=0x7f19dc1574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19c802f310 tx=0x7f19c80047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 shutdown_connections 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f19ac078e00 0x7f19ac07b2c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f19dc155a50 0x7f19dc06e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f19dc0c9160 0x7f19dc1f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 --2- 10.20.193.18:0/3124514010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f19dc072e50 0x7f19dc1574a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 >> 10.20.193.18:0/3124514010 conn(0x7f19dc0cd480 msgr2=0x7f19dc0c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 shutdown_connections 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 -- 10.20.193.18:0/3124514010 wait complete. 2026-02-24T18:35:36.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.587+0000 7f19e1217640 1 librados: shutdown 2026-02-24T18:35:36.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.2 2026-02-24T18:35:36.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.660+0000 7f0703c96640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:36.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.660+0000 7f0703c96640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:36.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.660+0000 7f0703c96640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:36.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.660+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f06fc158c90 con 0x7f06fc0c99a0 2026-02-24T18:35:36.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.664+0000 7f7d073ce640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:36.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.664+0000 7f7d073ce640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:36.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.664+0000 7f7d073ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.664+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7d001509b0 con 0x7f7d0015d730 2026-02-24T18:35:36.667 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:35:36.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.666+0000 7f7cedffb640 1 -- 10.20.193.18:0/2085884724 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f7cfc053390 con 0x7f7d0015d730 2026-02-24T18:35:36.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.666+0000 7f7cedffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:36.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.666+0000 7f7cedffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:36.668 INFO:teuthology.orchestra.run.trial018.stdout:90194313219 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 10 librados: watch_flush enter 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 10 librados: watch_flush exit 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 20 client.4516.objecter shutdown clearing up homeless session... 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 10 client.4516.objecter successfully canceled tick 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 msgr2=0x7f7cd007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 0x7f7cd007b2c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098a0 tx=0x7f7cf0006d90 comp rx=0 tx=0).stop 2026-02-24T18:35:36.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 10 monclient: shutdown 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 msgr2=0x7f7d000c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc02f2c0 tx=0x7f7cfc004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 shutdown_connections 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cd0078e00 0x7f7cd007b2c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d0015d730 0x7f7d000c8f70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d0015d2c0 0x7f7d001f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 --2- 10.20.193.18:0/2085884724 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d001527f0 0x7f7d000c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 >> 10.20.193.18:0/2085884724 conn(0x7f7d000ce730 msgr2=0x7f7d00150ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 shutdown_connections 2026-02-24T18:35:36.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 -- 10.20.193.18:0/2085884724 wait complete. 2026-02-24T18:35:36.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.668+0000 7f7d073ce640 1 librados: shutdown 2026-02-24T18:35:36.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.10 2026-02-24T18:35:36.686 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-24T18:35:36.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 10 monclient: monmap: 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.735200+0000 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.735200+0000 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.736 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.737 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.737 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.737 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.737 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.737 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.739 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 1 Processor -- start 2026-02-24T18:35:36.739 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 1 -- start start 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.734+0000 7f115880b640 10 monclient: init 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient: _add_conns ranks=[4,1,0,5,3,2] 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f11501527f0 0x7f1150073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): picked mon.noname-b con 0x7f11501527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f115015d2c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f115015d730 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1150073c20 con 0x7f115015d730 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1150073f00 con 0x7f115015d2c0 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1150074b10 con 0x7f11501527f0 2026-02-24T18:35:36.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f115880b640 10 monclient(hunting): authenticate will time out at 1480.416436s 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f1156d81640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f115015d730 0x7f11500ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58684/0 (socket says 10.20.193.18:58684) 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.735+0000 7f1156d81640 1 -- 10.20.193.18:0/442675220 learned_addr learned my addr 10.20.193.18:0/442675220 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.736+0000 7f1156580640 1 --2- 10.20.193.18:0/442675220 >> v2:10.20.193.32:3300/0 conn(0x7f11501527f0 0x7f1150073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.736+0000 7f1156580640 10 monclient(hunting): get_auth_request con 0x7f11501527f0 auth_method 0 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.736+0000 7f1156580640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.736+0000 7f1156580640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.736+0000 7f1156580640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 1 -- 10.20.193.18:0/442675220 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657335319 0 0) 0x7f1150073c20 con 0x7f115015d730 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 10 monclient(hunting): _negotiate 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 10 monclient(hunting): my global_id is 4581 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f115557e640 1 -- 10.20.193.18:0/442675220 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1140003620 con 0x7f115015d730 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.737+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 10 monclient(hunting): handle_auth_done global_id 4543 payload 274 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 1 -- 10.20.193.18:0/442675220 >> v1:10.20.193.32:6789/0 conn(0x7f115015d2c0 legacy=0x7f115015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 1 -- 10.20.193.18:0/442675220 >> v1:10.20.193.18:6789/0 conn(0x7f115015d730 legacy=0x7f11500ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 1 monclient: found mon.noname-b 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.738+0000 7f1156580640 1 -- 10.20.193.18:0/442675220 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11500741e0 con 0x7f11501527f0 2026-02-24T18:35:36.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.739+0000 7f1156580640 1 --2- 10.20.193.18:0/442675220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11501527f0 0x7f1150073530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11480098a0 tx=0x7f114802ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f4c8328f3713b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.740+0000 7f115557e640 1 -- 10.20.193.18:0/442675220 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114803c070 con 0x7f11501527f0 2026-02-24T18:35:36.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.740+0000 7f115557e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: dump: 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.748 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 5 monclient: authenticate success, global_id 4543 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 1 -- 10.20.193.18:0/442675220 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114802fb70 con 0x7f11501527f0 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 1 -- 10.20.193.18:0/442675220 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148037440 con 0x7f11501527f0 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115557e640 10 monclient: dump: 2026-02-24T18:35:36.749 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 4 set_mon_vals no callback set 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 10 monclient: shutdown 2026-02-24T18:35:36.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 -- 10.20.193.18:0/442675220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11501527f0 msgr2=0x7f1150073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 --2- 10.20.193.18:0/442675220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11501527f0 0x7f1150073530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11480098a0 tx=0x7f114802ee60 comp rx=0 tx=0).stop 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 -- 10.20.193.18:0/442675220 shutdown_connections 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 --2- 10.20.193.18:0/442675220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11501527f0 0x7f1150073530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 -- 10.20.193.18:0/442675220 >> 10.20.193.18:0/442675220 conn(0x7f11500ce730 msgr2=0x7f11500ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 -- 10.20.193.18:0/442675220 shutdown_connections 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.741+0000 7f115880b640 1 -- 10.20.193.18:0/442675220 wait complete. 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 10 monclient: monmap: 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.751 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 librados: starting msgr at 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 librados: starting objecter 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 Processor -- start 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 -- start start 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 librados: setting wanted keys 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 1 librados: calling monclient init 2026-02-24T18:35:36.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.742+0000 7f115880b640 10 monclient: init 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 0x7f115014c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): picked mon.noname-a con 0x7f11501527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 0x7f11500ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): picked mon.noname-b con 0x7f115015d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f115015d730 0x7f11501f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): picked mon.noname-c con 0x7f115015d730 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1150074fa0 con 0x7f11501527f0 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11500747a0 con 0x7f115015d2c0 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1150075120 con 0x7f115015d730 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f115880b640 10 monclient(hunting): authenticate will time out at 1480.424206s 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 0x7f11500ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1156580640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 0x7f115014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 0x7f11500ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53610/0 (socket says 10.20.193.18:53610) 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 1 -- 10.20.193.18:0/4228483896 learned_addr learned my addr 10.20.193.18:0/4228483896 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 10 monclient(hunting): get_auth_request con 0x7f115015d2c0 auth_method 0 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1155d7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1156580640 10 monclient(hunting): get_auth_request con 0x7f11501527f0 auth_method 0 2026-02-24T18:35:36.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1156580640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1156580640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.743+0000 7f1156580640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f115015d730 0x7f11501f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1155d7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1155d7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1155d7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156580640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): get_auth_request con 0x7f115015d730 auth_method 0 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.744+0000 7f1156d81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 10 monclient(hunting): handle_auth_done global_id 4584 payload 824 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 -- 10.20.193.18:0/4228483896 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f115015d730 msgr2=0x7f11501f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f115015d730 0x7f11501f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 -- 10.20.193.18:0/4228483896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 msgr2=0x7f11500ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 0x7f11500ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.745+0000 7f1156580640 1 -- 10.20.193.18:0/4228483896 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11500c9010 con 0x7f11501527f0 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.746+0000 7f1156580640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 0x7f115014c790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1148036000 tx=0x7f11480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.746+0000 7f113affd640 10 client.?.objecter ms_handle_connect 0x7f11501527f0 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.746+0000 7f113affd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148046020 con 0x7f11501527f0 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: dump: 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1148003a50 con 0x7f11501527f0 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 5 monclient: authenticate success, global_id 4584 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 10 monclient: _renew_subs 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11500c92b0 con 0x7f11501527f0 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114803c040 con 0x7f11501527f0 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f113affd640 10 monclient: dump: 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.761 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f1154d7d640 4 set_mon_vals no callback set 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 10 client.4584.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 10 monclient: _renew_subs 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11500c9870 con 0x7f11501527f0 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.748+0000 7f115880b640 1 librados: init done 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.749+0000 7f115880b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.749+0000 7f115880b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.749+0000 7f115880b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.749+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1150152bd0 con 0x7f11501527f0 2026-02-24T18:35:36.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.762+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1148042430 con 0x7f11501527f0 2026-02-24T18:35:36.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.762+0000 7f113affd640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 0x7f112007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.762+0000 7f1155d7f640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 0x7f112007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.762+0000 7f1155d7f640 10 monclient: get_auth_request con 0x7f1120078e80 auth_method 0 2026-02-24T18:35:36.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.763+0000 7f113affd640 10 client.4584.objecter ms_handle_connect 0x7f1120078e80 2026-02-24T18:35:36.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.763+0000 7f1155d7f640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 0x7f112007b340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11501f4300 tx=0x7f113c002a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.764+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f1148003e00 con 0x7f11501527f0 2026-02-24T18:35:36.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.764+0000 7f113affd640 10 client.4584.objecter ms_dispatch2 0x7f1150000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.764+0000 7f113affd640 3 client.4584.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.764+0000 7f113affd640 3 client.4584.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.764+0000 7f113affd640 20 client.4584.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 10 monclient: monmap: 2026-02-24T18:35:36.784 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.784 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.783982+0000 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.783982+0000 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.785 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 1 Processor -- start 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 1 -- start start 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.783+0000 7fa9598f3640 10 monclient: init 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient: _add_conns ranks=[0,2,1,4,5,3] 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9541527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa954152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa9540c6190 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95406e300 con 0x7fa9541527d0 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95406e5e0 con 0x7fa9540c6190 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95406e8c0 con 0x7fa954152bb0 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9598f3640 10 monclient(hunting): authenticate will time out at 1480.465204s 2026-02-24T18:35:36.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9537fe640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fa9540c6190 0x7fa9540c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51348/0 (socket says 10.20.193.18:51348) 2026-02-24T18:35:36.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.784+0000 7fa9537fe640 1 -- 10.20.193.18:0/852417637 learned_addr learned my addr 10.20.193.18:0/852417637 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:36.841 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-24T18:35:36.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.865+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1148050020 con 0x7f11501527f0 2026-02-24T18:35:36.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.865+0000 7f113affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.865+0000 7f113affd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987641464 0 0) 0x7fa95406e300 con 0x7fa9541527d0 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 10 monclient(hunting): my global_id is 4587 2026-02-24T18:35:36.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.868+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa940003620 con 0x7fa9541527d0 2026-02-24T18:35:36.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1208150766 0 0) 0x7fa940003620 con 0x7fa9541527d0 2026-02-24T18:35:36.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 >> v1:10.20.193.110:6789/0 conn(0x7fa954152bb0 legacy=0x7fa9540c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 >> v1:10.20.193.32:6789/0 conn(0x7fa9540c6190 legacy=0x7fa9540c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95406eba0 con 0x7fa9541527d0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7fa948002f00 con 0x7fa9541527d0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: dump: 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.876 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 >> v1:10.20.193.18:6789/0 conn(0x7fa9541527d0 legacy=0x7fa95406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa940003c20 0x7fa940004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): picked mon.c con 0x7fa940003c20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 0x7fa940004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): picked mon.b con 0x7fa940004900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940009ab0 0x7fa94000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): picked mon.a con 0x7fa940009ab0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa94000c4c0 con 0x7fa940009ab0 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa94000c670 con 0x7fa940004900 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa94000c870 con 0x7fa940003c20 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.871+0000 7fa951ffb640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 0x7fa940004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa940003c20 0x7fa940004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9537fe640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940009ab0 0x7fa94000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): get_auth_request con 0x7fa940004900 auth_method 0 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): get_auth_request con 0x7fa940003c20 auth_method 0 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9537fe640 10 monclient(hunting): get_auth_request con 0x7fa940009ab0 auth_method 0 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9537fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:36.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.872+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 10 monclient(hunting): handle_auth_done global_id 4587 payload 293 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 -- 10.20.193.18:0/852417637 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa940003c20 msgr2=0x7fa940004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa940003c20 0x7fa940004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 -- 10.20.193.18:0/852417637 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940009ab0 msgr2=0x7fa94000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940009ab0 0x7fa94000bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 monclient: found mon.b 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9537fe640 1 -- 10.20.193.18:0/852417637 reap_dead start 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 1 -- 10.20.193.18:0/852417637 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa940003620 con 0x7fa940004900 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9598f3640 5 monclient: authenticate success, global_id 4587 2026-02-24T18:35:36.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.873+0000 7fa9598f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:36.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.874+0000 7fa9527fc640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 0x7fa940004d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa93c002c80 tx=0x7fa93c031060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d998132db7d343a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c002cc0 con 0x7fa940004900 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: dump: 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.887 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c0052d0 con 0x7fa940004900 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 1 -- 10.20.193.18:0/852417637 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0318c0 con 0x7fa940004900 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa951ffb640 10 monclient: dump: 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.888 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 4 set_mon_vals no callback set 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 10 monclient: shutdown 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 -- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 msgr2=0x7fa940004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 0x7fa940004d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa93c002c80 tx=0x7fa93c031060 comp rx=0 tx=0).stop 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 -- 10.20.193.18:0/852417637 shutdown_connections 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 --2- 10.20.193.18:0/852417637 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa940004900 0x7fa940004d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 -- 10.20.193.18:0/852417637 >> 10.20.193.18:0/852417637 conn(0x7fa9540cd780 msgr2=0x7fa9540cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 -- 10.20.193.18:0/852417637 shutdown_connections 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.876+0000 7fa9598f3640 1 -- 10.20.193.18:0/852417637 wait complete. 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient: monmap: 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.890 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 librados: starting msgr at 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 librados: starting objecter 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 Processor -- start 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 -- start start 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 librados: setting wanted keys 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 librados: calling monclient init 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient: init 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940004900 0x7fa9541ea090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-a con 0x7fa940004900 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa954157790 0x7fa9541ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.877+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-c con 0x7fa954157790 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): picked mon.noname-b con 0x7fa9540bc8f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540c5a80 con 0x7fa940004900 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa95406f1f0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540c5c00 con 0x7fa954157790 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9598f3640 10 monclient(hunting): authenticate will time out at 1480.559024s 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940004900 0x7fa9541ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53634/0 (socket says 10.20.193.18:53634) 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 1 -- 10.20.193.18:0/3113133982 learned_addr learned my addr 10.20.193.18:0/3113133982 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): get_auth_request con 0x7fa9540bc8f0 auth_method 0 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): get_auth_request con 0x7fa940004900 auth_method 0 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9527fc640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa954157790 0x7fa9541ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa952ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9527fc640 10 monclient(hunting): get_auth_request con 0x7fa954157790 auth_method 0 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9527fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.878+0000 7fa9527fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 10 monclient(hunting): handle_auth_done global_id 4549 payload 824 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 -- 10.20.193.18:0/3113133982 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa954157790 msgr2=0x7fa9541ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa954157790 0x7fa9541ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 -- 10.20.193.18:0/3113133982 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940004900 msgr2=0x7fa9541ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940004900 0x7fa9541ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 monclient: found mon.noname-b 2026-02-24T18:35:36.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.879+0000 7fa9537fe640 1 -- 10.20.193.18:0/3113133982 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9540bd2b0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.881+0000 7fa9537fe640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9540c55f0 tx=0x7fa94400f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.881+0000 7fa92bfff640 10 client.?.objecter ms_handle_connect 0x7fa9540bc8f0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.881+0000 7fa92bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94401a070 con 0x7fa9540bc8f0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: dump: 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.895 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94400fe50 con 0x7fa9540bc8f0 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944012570 con 0x7fa9540bc8f0 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:36.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9517fa640 4 set_mon_vals no callback set 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa92bfff640 10 monclient: dump: 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 5 monclient: authenticate success, global_id 4549 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 10 monclient: _renew_subs 2026-02-24T18:35:36.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9540bc3f0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 10 client.4549.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 10 monclient: _renew_subs 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa954157fa0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.882+0000 7fa9598f3640 1 librados: init done 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.883+0000 7fa9598f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.883+0000 7fa9598f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.883+0000 7fa9598f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:36.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.883+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9540bc5c0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.895+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa944012710 con 0x7fa9540bc8f0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.895+0000 7fa92bfff640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 0x7fa92407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.895+0000 7fa952ffd640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 0x7fa92407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.895+0000 7fa952ffd640 10 monclient: get_auth_request con 0x7fa924078e30 auth_method 0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa952ffd640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 0x7fa92407b2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa954064240 tx=0x7fa948009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 10 client.4549.objecter ms_handle_connect 0x7fa924078e30 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa94405d7d0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 10 client.4549.objecter ms_dispatch2 0x7fa954000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 3 client.4549.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 3 client.4549.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:36.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.896+0000 7fa92bfff640 20 client.4549.objecter dump_active .. 0 homeless 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.909+0000 7f7eb4bfe640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.909+0000 7f7eb4bfe640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.909+0000 7f7eb4bfe640 10 monclient: monmap: 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.910773+0000 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.910773+0000 2026-02-24T18:35:36.911 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.912 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.910+0000 7f7eb4bfe640 1 Processor -- start 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.910+0000 7f7eb4bfe640 1 -- start start 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.910+0000 7f7eb4bfe640 10 monclient: init 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient: _add_conns ranks=[1,4,3,2,0,5] 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7eb00c90d0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7eb00c94b0 0x7f7eb0158950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-b con 0x7f7eb00c94b0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7eb0158e90 0x7f7eb0074250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-a con 0x7f7eb0158e90 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb0073300 con 0x7f7eb00c90d0 2026-02-24T18:35:36.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb015ce70 con 0x7f7eb0158e90 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb0073540 con 0x7f7eb00c94b0 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eb4bfe640 10 monclient(hunting): authenticate will time out at 1480.592171s 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7eb0158e90 0x7f7eb0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7eb0158e90 0x7f7eb0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41908/0 (socket says 10.20.193.18:41908) 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 1 -- 10.20.193.18:0/3234141780 learned_addr learned my addr 10.20.193.18:0/3234141780 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 10 monclient(hunting): get_auth_request con 0x7f7eb0158e90 auth_method 0 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.911+0000 7f7eaed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.912+0000 7f7eaed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.912+0000 7f7eaed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.912+0000 7f7eaed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.916 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 10 monclient(hunting): handle_auth_done global_id 4593 payload 274 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 1 -- 10.20.193.18:0/3234141780 >> v2:10.20.193.32:3300/0 conn(0x7f7eb00c94b0 msgr2=0x7f7eb0158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 1 --2- 10.20.193.18:0/3234141780 >> v2:10.20.193.32:3300/0 conn(0x7f7eb00c94b0 0x7f7eb0158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 1 -- 10.20.193.18:0/3234141780 >> v1:10.20.193.32:6789/0 conn(0x7f7eb00c90d0 legacy=0x7f7eb0072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.913+0000 7f7eaed76640 1 -- 10.20.193.18:0/3234141780 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0159440 con 0x7f7eb0158e90 2026-02-24T18:35:36.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.914+0000 7f7eaed76640 1 --2- 10.20.193.18:0/3234141780 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb0158e90 0x7f7eb0074250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0072480 tx=0x7f7ea002f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba46b88208e4cf7 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 1 -- 10.20.193.18:0/3234141780 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea002fcd0 con 0x7f7eb0158e90 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: dump: 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.920 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 1 -- 10.20.193.18:0/3234141780 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00373f0 con 0x7f7eb0158e90 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7eb4bfe640 5 monclient: authenticate success, global_id 4593 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7eb4bfe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 1 -- 10.20.193.18:0/3234141780 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00377c0 con 0x7f7eb0158e90 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7ead573640 10 monclient: dump: 2026-02-24T18:35:36.921 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7eb4bfe640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.916+0000 7f7eb4bfe640 4 set_mon_vals no callback set 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 10 monclient: shutdown 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/3234141780 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb0158e90 msgr2=0x7f7eb0074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/3234141780 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb0158e90 0x7f7eb0074250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0072480 tx=0x7f7ea002f270 comp rx=0 tx=0).stop 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/3234141780 shutdown_connections 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/3234141780 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb0158e90 0x7f7eb0074250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/3234141780 >> v2:10.20.193.32:3300/0 conn(0x7f7eb00c94b0 0x7f7eb0158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/3234141780 >> 10.20.193.18:0/3234141780 conn(0x7f7eb00cd780 msgr2=0x7f7eb00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/3234141780 shutdown_connections 2026-02-24T18:35:36.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/3234141780 wait complete. 2026-02-24T18:35:36.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 10 monclient: monmap: 2026-02-24T18:35:36.927 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.927 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:36.927 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 librados: starting msgr at 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 librados: starting objecter 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.917+0000 7f7eb4bfe640 1 Processor -- start 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 -- start start 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 librados: setting wanted keys 2026-02-24T18:35:36.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 librados: calling monclient init 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient: init 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7eb00c90d0 0x7f7eb01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-c con 0x7f7eb00c90d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-a con 0x7f7eb00c94b0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7eb0158e90 0x7f7eb0142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): picked mon.noname-b con 0x7f7eb0158e90 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0073a70 con 0x7f7eb00c94b0 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0073790 con 0x7f7eb0158e90 2026-02-24T18:35:36.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0073d50 con 0x7f7eb00c90d0 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eb4bfe640 10 monclient(hunting): authenticate will time out at 1480.599435s 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41920/0 (socket says 10.20.193.18:41920) 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 1 -- 10.20.193.18:0/884937246 learned_addr learned my addr 10.20.193.18:0/884937246 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 10 monclient(hunting): get_auth_request con 0x7f7eb00c94b0 auth_method 0 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eadd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.918+0000 7f7eae575640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7eb00c90d0 0x7f7eb01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eadd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eadd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eadd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): get_auth_request con 0x7f7eb00c90d0 auth_method 0 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.919+0000 7f7eae575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 10 monclient(hunting): handle_auth_done global_id 4596 payload 824 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 -- 10.20.193.18:0/884937246 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7eb00c90d0 msgr2=0x7f7eb01f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7eb00c90d0 0x7f7eb01f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 -- 10.20.193.18:0/884937246 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7eb0158e90 msgr2=0x7f7eb0142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7eb0158e90 0x7f7eb0142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 10 monclient: _finish_hunting 0 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 monclient: found mon.noname-a 2026-02-24T18:35:36.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.920+0000 7f7eadd74640 1 -- 10.20.193.18:0/884937246 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01f1d60 con 0x7f7eb00c94b0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.921+0000 7f7eadd74640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b780 tx=0x7f7e9800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.921+0000 7f7e96ffd640 10 client.?.objecter ms_handle_connect 0x7f7eb00c94b0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.921+0000 7f7e96ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004480 con 0x7f7eb00c94b0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: dump: 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.932 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98010070 con 0x7f7eb00c94b0 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98002920 con 0x7f7eb00c94b0 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 5 monclient: authenticate success, global_id 4596 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eacd72640 4 set_mon_vals no callback set 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 10 monclient: _renew_subs 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01f2030 con 0x7f7eb00c94b0 2026-02-24T18:35:36.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7e96ffd640 10 monclient: dump: 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 10 client.4596.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 10 monclient: _renew_subs 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01f25f0 con 0x7f7eb00c94b0 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.923+0000 7f7eb4bfe640 1 librados: init done 2026-02-24T18:35:36.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.924+0000 7f7eb4bfe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.924+0000 7f7eb4bfe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.924+0000 7f7eb4bfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.924+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb0158950 con 0x7f7eb00c94b0 2026-02-24T18:35:36.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.952+0000 7f115880b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:36.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.952+0000 7f115880b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:36.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.952+0000 7f115880b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:36.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.952+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f11500cd980 con 0x7f11501527f0 2026-02-24T18:35:36.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 10 monclient: get_monmap_and_config 2026-02-24T18:35:36.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 10 monclient: build_initial_monmap 2026-02-24T18:35:36.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 10 monclient: monmap: 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:36.985199+0000 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:36.985199+0000 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:36.986 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:36.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 1 Processor -- start 2026-02-24T18:35:36.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 1 -- start start 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.984+0000 7ffbb4ecd640 10 monclient: init 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient: _add_conns ranks=[3,0,4,1,5,2] 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ffbb006ff20 0x7ffbb0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-a con 0x7ffbb006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffbb00c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ffbb015a5e0 0x7ffbb014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-b con 0x7ffbb015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb00c7e90 con 0x7ffbb00c99a0 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb0159a40 con 0x7ffbb006ff20 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:36.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb00c8170 con 0x7ffbb015a5e0 2026-02-24T18:35:36.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:36.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.985+0000 7ffbb4ecd640 10 monclient(hunting): authenticate will time out at 1480.666330s 2026-02-24T18:35:36.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.995+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa9540bd2b0 con 0x7fa9540bc8f0 2026-02-24T18:35:36.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.995+0000 7fa92bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:36.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.995+0000 7fa92bfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:36.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ffbb015a5e0 0x7ffbb014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ffbb015a5e0 0x7ffbb014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53662/0 (socket says 10.20.193.18:53662) 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 1 -- 10.20.193.18:0/2471386796 learned_addr learned my addr 10.20.193.18:0/2471386796 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 10 monclient(hunting): get_auth_request con 0x7ffbb015a5e0 auth_method 0 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:36.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.996+0000 7ffbaed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:36.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.997+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:36.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.997+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:36.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 10 monclient(hunting): handle_auth_done global_id 4552 payload 274 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 1 -- 10.20.193.18:0/2471386796 >> v2:10.20.193.18:3300/0 conn(0x7ffbb006ff20 msgr2=0x7ffbb0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 1 --2- 10.20.193.18:0/2471386796 >> v2:10.20.193.18:3300/0 conn(0x7ffbb006ff20 0x7ffbb0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 1 -- 10.20.193.18:0/2471386796 >> v1:10.20.193.18:6789/0 conn(0x7ffbb00c99a0 legacy=0x7ffbb0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.998+0000 7ffbaed76640 1 -- 10.20.193.18:0/2471386796 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb00c8450 con 0x7ffbb015a5e0 2026-02-24T18:35:37.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:36.999+0000 7ffbaed76640 1 --2- 10.20.193.18:0/2471386796 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb015a5e0 0x7ffbb014fe10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009e50 tx=0x7ffba002f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8202f3989783c47c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 1 -- 10.20.193.18:0/2471386796 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba002fcc0 con 0x7ffbb015a5e0 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: dump: 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.006 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 1 -- 10.20.193.18:0/2471386796 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba00373f0 con 0x7ffbb015a5e0 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbb4ecd640 5 monclient: authenticate success, global_id 4552 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbb4ecd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 1 -- 10.20.193.18:0/2471386796 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba002fe60 con 0x7ffbb015a5e0 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbad573640 10 monclient: dump: 2026-02-24T18:35:37.007 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbb4ecd640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.001+0000 7ffbb4ecd640 4 set_mon_vals no callback set 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 10 monclient: shutdown 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2471386796 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb015a5e0 msgr2=0x7ffbb014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2471386796 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb015a5e0 0x7ffbb014fe10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009e50 tx=0x7ffba002f2a0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2471386796 shutdown_connections 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2471386796 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb015a5e0 0x7ffbb014fe10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2471386796 >> v2:10.20.193.18:3300/0 conn(0x7ffbb006ff20 0x7ffbb0070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2471386796 >> 10.20.193.18:0/2471386796 conn(0x7ffbb00cd780 msgr2=0x7ffbb00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2471386796 shutdown_connections 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2471386796 wait complete. 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 10 monclient: monmap: 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.009 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 librados: starting msgr at 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 librados: starting objecter 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 Processor -- start 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 -- start start 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 librados: setting wanted keys 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 1 librados: calling monclient init 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.002+0000 7ffbb4ecd640 10 monclient: init 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:37.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-b con 0x7ffbb006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbb00c99a0 0x7ffbb00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-a con 0x7ffbb00c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbb015a5e0 0x7ffbb01f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): picked mon.noname-c con 0x7ffbb015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0150350 con 0x7ffbb00c99a0 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb00c8a10 con 0x7ffbb006ff20 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb01504d0 con 0x7ffbb015a5e0 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbb4ecd640 10 monclient(hunting): authenticate will time out at 1480.684166s 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53670/0 (socket says 10.20.193.18:53670) 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 1 -- 10.20.193.18:0/2561662353 learned_addr learned my addr 10.20.193.18:0/2561662353 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): get_auth_request con 0x7ffbb006ff20 auth_method 0 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbaed76640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbb015a5e0 0x7ffbb01f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbae575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbaed76640 10 monclient(hunting): get_auth_request con 0x7ffbb015a5e0 auth_method 0 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbaed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbaed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.003+0000 7ffbaed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.004+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.004+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.004+0000 7ffbaed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.013 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-24T18:35:37.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 10 monclient(hunting): handle_auth_done global_id 4555 payload 824 2026-02-24T18:35:37.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 -- 10.20.193.18:0/2561662353 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbb015a5e0 msgr2=0x7ffbb01f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbb015a5e0 0x7ffbb01f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 -- 10.20.193.18:0/2561662353 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbb00c99a0 msgr2=0x7ffbb00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbb00c99a0 0x7ffbb00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.005+0000 7ffbae575640 1 -- 10.20.193.18:0/2561662353 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb00cc000 con 0x7ffbb006ff20 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.006+0000 7ffbae575640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b7e0 tx=0x7ffba400bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.006+0000 7ffb96ffd640 10 client.?.objecter ms_handle_connect 0x7ffbb006ff20 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.006+0000 7ffb96ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.006+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4004480 con 0x7ffbb006ff20 2026-02-24T18:35:37.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.006+0000 7ffb96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: dump: 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 5 monclient: authenticate success, global_id 4555 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _renew_subs 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb00cc2e0 con 0x7ffbb006ff20 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 client.4555.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _renew_subs 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb00cc8a0 con 0x7ffbb006ff20 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 1 librados: init done 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb01508a0 con 0x7ffbb006ff20 2026-02-24T18:35:37.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba4010070 con 0x7ffbb006ff20 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba40028e0 con 0x7ffbb006ff20 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffb96ffd640 10 monclient: dump: 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.019 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.020 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.007+0000 7ffbacd72640 4 set_mon_vals no callback set 2026-02-24T18:35:37.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.031+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7e9801d030 con 0x7f7eb00c94b0 2026-02-24T18:35:37.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.032+0000 7f7e96ffd640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 0x7f7e7c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.032+0000 7f7eae575640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 0x7f7e7c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.032+0000 7f7eae575640 10 monclient: get_auth_request con 0x7f7e7c078e50 auth_method 0 2026-02-24T18:35:37.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.032+0000 7f7e96ffd640 10 client.4596.objecter ms_handle_connect 0x7f7e7c078e50 2026-02-24T18:35:37.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.032+0000 7f7eae575640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 0x7f7e7c07b310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4004660 tx=0x7f7ea4009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f7e9805d290 con 0x7f7eb00c94b0 2026-02-24T18:35:37.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 10 client.4596.objecter ms_dispatch2 0x7f7eb0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 3 client.4596.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 3 client.4596.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 20 client.4596.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.040+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7e98024080 con 0x7f7eb00c94b0 2026-02-24T18:35:37.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.041+0000 7f7e96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.041+0000 7f7e96ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.046+0000 7f113affd640 1 -- 10.20.193.18:0/4228483896 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1148087080 con 0x7f11501527f0 2026-02-24T18:35:37.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.046+0000 7f113affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:37.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.046+0000 7f113affd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.048 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 10 librados: watch_flush enter 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 10 librados: watch_flush exit 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 20 client.4584.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 10 client.4584.objecter successfully canceled tick 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 msgr2=0x7f112007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 0x7f112007b340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11501f4300 tx=0x7f113c002a30 comp rx=0 tx=0).stop 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 10 monclient: shutdown 2026-02-24T18:35:37.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 msgr2=0x7f115014c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 0x7f115014c790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1148036000 tx=0x7f11480047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 shutdown_connections 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1120078e80 0x7f112007b340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f115015d2c0 0x7f11500ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f115015d730 0x7f11501f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 --2- 10.20.193.18:0/4228483896 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f11501527f0 0x7f115014c790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 >> 10.20.193.18:0/4228483896 conn(0x7f11500ce730 msgr2=0x7f11500cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 shutdown_connections 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 -- 10.20.193.18:0/4228483896 wait complete. 2026-02-24T18:35:37.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.049+0000 7f115880b640 1 librados: shutdown 2026-02-24T18:35:37.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.057+0000 7f06ea7fc640 1 -- 10.20.193.18:0/1625320873 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06f0061b70 con 0x7f06fc0c99a0 2026-02-24T18:35:37.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.057+0000 7f06ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:37.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.057+0000 7f06ea7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.059 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:37.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.0 2026-02-24T18:35:37.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 10 librados: watch_flush enter 2026-02-24T18:35:37.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 10 librados: watch_flush exit 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 20 client.4529.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 10 client.4529.objecter successfully canceled tick 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 msgr2=0x7f06cc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 0x7f06cc07b2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06ec039000 tx=0x7f06ec031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 10 monclient: shutdown 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 msgr2=0x7f06fc0bb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c760 tx=0x7f06f000cc30 comp rx=0 tx=0).stop 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 shutdown_connections 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f06cc078e00 0x7f06cc07b2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f06fc15a5e0 0x7f06fc0bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f06fc0c99a0 0x7f06fc0bb540 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 --2- 10.20.193.18:0/1625320873 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f06fc06ff20 0x7f06fc06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 >> 10.20.193.18:0/1625320873 conn(0x7f06fc0cd780 msgr2=0x7f06fc073090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 shutdown_connections 2026-02-24T18:35:37.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 -- 10.20.193.18:0/1625320873 wait complete. 2026-02-24T18:35:37.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.059+0000 7f0703c96640 1 librados: shutdown 2026-02-24T18:35:37.063 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-24T18:35:37.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.7 2026-02-24T18:35:37.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.082+0000 7fa9598f3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:37.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.082+0000 7fa9598f3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:37.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.082+0000 7fa9598f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.082+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa95406da60 con 0x7fa9540bc8f0 2026-02-24T18:35:37.090 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 10 monclient: monmap: 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.091325+0000 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.091325+0000 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.091 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 1 Processor -- start 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 1 -- start start 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.090+0000 7f67d2511640 10 monclient: init 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient: _add_conns ranks=[4,5,0,3,2,1] 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f67cc1527f0 0x7f67cc073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-b con 0x7f67cc1527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f67cc15d2c0 0x7f67cc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-c con 0x7f67cc15d2c0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f67cc15d730 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67cc073c20 con 0x7f67cc15d730 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc074b10 con 0x7f67cc1527f0 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc073f00 con 0x7f67cc15d2c0 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.091+0000 7f67d2511640 10 monclient(hunting): authenticate will time out at 1480.772549s 2026-02-24T18:35:37.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f67cc15d2c0 0x7f67cc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f67cc15d2c0 0x7f67cc15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44620/0 (socket says 10.20.193.18:44620) 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 1 -- 10.20.193.18:0/3142074581 learned_addr learned my addr 10.20.193.18:0/3142074581 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 10 monclient(hunting): get_auth_request con 0x7f67cc15d2c0 auth_method 0 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.108+0000 7f67cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.109+0000 7f67cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.109+0000 7f67cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.109+0000 7f67cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.109+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ffba401d030 con 0x7ffbb006ff20 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 10 monclient(hunting): handle_auth_done global_id 4547 payload 274 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 1 -- 10.20.193.18:0/3142074581 >> v2:10.20.193.32:3300/0 conn(0x7f67cc1527f0 msgr2=0x7f67cc073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 1 --2- 10.20.193.18:0/3142074581 >> v2:10.20.193.32:3300/0 conn(0x7f67cc1527f0 0x7f67cc073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 1 -- 10.20.193.18:0/3142074581 >> v1:10.20.193.18:6789/0 conn(0x7f67cc15d730 legacy=0x7f67cc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7f67cb7fe640 1 -- 10.20.193.18:0/3142074581 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc0741e0 con 0x7f67cc15d2c0 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7ffb96ffd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 0x7ffb7c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7ffbadd74640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 0x7ffb7c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.110+0000 7ffbadd74640 10 monclient: get_auth_request con 0x7ffb7c078e00 auth_method 0 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffbadd74640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 0x7ffb7c07b2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 10 client.4555.objecter ms_handle_connect 0x7ffb7c078e00 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7f67cb7fe640 1 --2- 10.20.193.18:0/3142074581 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc15d2c0 0x7f67cc15f3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67cc08f510 tx=0x7f67bc02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7403f9f61210e25b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7ffba4096c60 con 0x7ffbb006ff20 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 10 client.4555.objecter ms_dispatch2 0x7ffbb0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 3 client.4555.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 3 client.4555.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.111+0000 7ffb96ffd640 20 client.4555.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 1 -- 10.20.193.18:0/3142074581 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc03c070 con 0x7f67cc15d2c0 2026-02-24T18:35:37.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: dump: 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 1 -- 10.20.193.18:0/3142074581 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc02fb40 con 0x7f67cc15d2c0 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 1 -- 10.20.193.18:0/3142074581 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc037440 con 0x7f67cc15d2c0 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67d2511640 5 monclient: authenticate success, global_id 4547 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67d2511640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67d2511640 4 set_mon_vals no callback set 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.112+0000 7f67caffd640 10 monclient: dump: 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.114 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 10 monclient: shutdown 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- 10.20.193.18:0/3142074581 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc15d2c0 msgr2=0x7f67cc15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 --2- 10.20.193.18:0/3142074581 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc15d2c0 0x7f67cc15f3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67cc08f510 tx=0x7f67bc02eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- 10.20.193.18:0/3142074581 shutdown_connections 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 --2- 10.20.193.18:0/3142074581 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc15d2c0 0x7f67cc15f3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 --2- 10.20.193.18:0/3142074581 >> v2:10.20.193.32:3300/0 conn(0x7f67cc1527f0 0x7f67cc073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- 10.20.193.18:0/3142074581 >> 10.20.193.18:0/3142074581 conn(0x7f67cc0ce730 msgr2=0x7f67cc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- 10.20.193.18:0/3142074581 shutdown_connections 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- 10.20.193.18:0/3142074581 wait complete. 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 10 monclient: monmap: 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 librados: starting msgr at 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 librados: starting objecter 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 Processor -- start 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 -- start start 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 librados: setting wanted keys 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 1 librados: calling monclient init 2026-02-24T18:35:37.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.113+0000 7f67d2511640 10 monclient: init 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-c con 0x7f67cc1527f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f67cc15d2c0 0x7f67cc1f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-b con 0x7f67cc15d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f67cc15d730 0x7f67cc1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): picked mon.noname-a con 0x7f67cc15d730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc074fa0 con 0x7f67cc15d730 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc0747a0 con 0x7f67cc15d2c0 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc075120 con 0x7f67cc1527f0 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67d2511640 10 monclient(hunting): authenticate will time out at 1480.795329s 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44632/0 (socket says 10.20.193.18:44632) 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 1 -- 10.20.193.18:0/2297947437 learned_addr learned my addr 10.20.193.18:0/2297947437 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 10 monclient(hunting): get_auth_request con 0x7f67cc1527f0 auth_method 0 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.114+0000 7f67cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.115+0000 7f67cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.115+0000 7f67cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.115+0000 7f67cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 10 monclient(hunting): handle_auth_done global_id 4550 payload 824 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 -- 10.20.193.18:0/2297947437 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f67cc15d2c0 msgr2=0x7f67cc1f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f67cc15d2c0 0x7f67cc1f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 -- 10.20.193.18:0/2297947437 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f67cc15d730 msgr2=0x7f67cc1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f67cc15d730 0x7f67cc1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.116+0000 7f67cbfff640 1 -- 10.20.193.18:0/2297947437 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc1f4520 con 0x7f67cc1527f0 2026-02-24T18:35:37.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.117+0000 7f67cbfff640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b7b0 tx=0x7f67b800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.117+0000 7f67c8ff9640 10 client.?.objecter ms_handle_connect 0x7f67cc1527f0 2026-02-24T18:35:37.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.117+0000 7f67c8ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004480 con 0x7f67cc1527f0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: dump: 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 5 monclient: authenticate success, global_id 4550 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 monclient: _renew_subs 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc1f4800 con 0x7f67cc1527f0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 client.4550.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 monclient: _renew_subs 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc0c58d0 con 0x7f67cc1527f0 2026-02-24T18:35:37.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 1 librados: init done 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8010070 con 0x7f67cc1527f0 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b80028e0 con 0x7f67cc1527f0 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67ca7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67c8ff9640 10 monclient: dump: 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.119+0000 7f67d2511640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.120+0000 7f67d2511640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.120+0000 7f67d2511640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.120+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67cc0734b0 con 0x7f67cc1527f0 2026-02-24T18:35:37.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.121+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffba4023020 con 0x7ffbb006ff20 2026-02-24T18:35:37.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.121+0000 7ffb96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.121+0000 7ffb96ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.128+0000 7f7eb4bfe640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:37.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f7eb4bfe640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:37.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f7eb4bfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7eb00c79d0 con 0x7f7eb00c94b0 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f0a1ec1a640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f0a1ec1a640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.129+0000 7f0a1ec1a640 10 monclient: monmap: 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.130811+0000 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.130811+0000 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.131 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.130+0000 7f0a1ec1a640 1 Processor -- start 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.130+0000 7f0a1ec1a640 1 -- start start 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.130+0000 7f0a1ec1a640 10 monclient: init 2026-02-24T18:35:37.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.130+0000 7f0a1ec1a640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.130+0000 7f0a1ec1a640 10 monclient: _add_conns ranks=[3,5,1,4,0,2] 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0a1815f650 0x7f0a1815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-a con 0x7f0a1815f650 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f0a180ceca0 0x7f0a180d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-c con 0x7f0a180ceca0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0a180d1ea0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a180d2450 con 0x7f0a180d1ea0 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0a180c8dd0 con 0x7f0a1815f650 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0a180d2730 con 0x7f0a180ceca0 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.131+0000 7f0a1ec1a640 10 monclient(hunting): authenticate will time out at 1480.812091s 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.132+0000 7f0a1d190640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f0a180d1ea0 0x7f0a180c9b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51362/0 (socket says 10.20.193.18:51362) 2026-02-24T18:35:37.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.132+0000 7f0a1d190640 1 -- 10.20.193.18:0/2760464922 learned_addr learned my addr 10.20.193.18:0/2760464922 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:37.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.134+0000 7f7e96ffd640 1 -- 10.20.193.18:0/884937246 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e98061950 con 0x7f7eb00c94b0 2026-02-24T18:35:37.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.134+0000 7f7e96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:37.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.134+0000 7f7e96ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.134+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f67b801d030 con 0x7f67cc1527f0 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stdout:90194313218 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.134+0000 7f67c8ff9640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 0x7f679c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f67cb7fe640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 0x7f679c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f67cb7fe640 10 monclient: get_auth_request con 0x7f679c078e00 auth_method 0 2026-02-24T18:35:37.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f67c8ff9640 10 client.4550.objecter ms_handle_connect 0x7f679c078e00 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f67cb7fe640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 0x7f679c07b2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67cc06cdf0 tx=0x7f67bc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 1 --2- 10.20.193.18:0/2760464922 >> v2:10.20.193.18:3300/0 conn(0x7f0a1815f650 0x7f0a1815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): get_auth_request con 0x7f0a1815f650 auth_method 0 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.135+0000 7f0a1c98f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 10 librados: watch_flush enter 2026-02-24T18:35:37.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 10 librados: watch_flush exit 2026-02-24T18:35:37.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 20 client.4596.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 10 client.4596.objecter successfully canceled tick 2026-02-24T18:35:37.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 msgr2=0x7f7e7c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 0x7f7e7c07b310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4004660 tx=0x7f7ea4009110 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 10 monclient: shutdown 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 msgr2=0x7f7eb01f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b780 tx=0x7f7e9800bc50 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 shutdown_connections 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7e7c078e50 0x7f7e7c07b310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7eb0158e90 0x7f7eb0142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7eb00c94b0 0x7f7eb01f1820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 --2- 10.20.193.18:0/884937246 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7eb00c90d0 0x7f7eb01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 >> 10.20.193.18:0/884937246 conn(0x7f7eb00cd780 msgr2=0x7f7eb00c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 shutdown_connections 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 -- 10.20.193.18:0/884937246 wait complete. 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.136+0000 7f7eb4bfe640 1 librados: shutdown 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 10 monclient(hunting): handle_auth_done global_id 4599 payload 274 2026-02-24T18:35:37.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 1 -- 10.20.193.18:0/2760464922 >> v2:10.20.193.110:3300/0 conn(0x7f0a180ceca0 msgr2=0x7f0a180d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 1 --2- 10.20.193.18:0/2760464922 >> v2:10.20.193.110:3300/0 conn(0x7f0a180ceca0 0x7f0a180d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 1 -- 10.20.193.18:0/2760464922 >> v1:10.20.193.32:6789/0 conn(0x7f0a180d1ea0 legacy=0x7f0a180c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 1 monclient: found mon.noname-a 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.137+0000 7f0a1c98f640 1 -- 10.20.193.18:0/2760464922 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a180c9020 con 0x7f0a1815f650 2026-02-24T18:35:37.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.138+0000 7f0a1c98f640 1 --2- 10.20.193.18:0/2760464922 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a1815f650 0x7f0a1815fa30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a180ce0a0 tx=0x7f0a0802ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3abff6b136eef21 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 1 -- 10.20.193.18:0/2760464922 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0803c070 con 0x7f0a1815f650 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: dump: 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 5 monclient: authenticate success, global_id 4599 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 1 -- 10.20.193.18:0/2760464922 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0802fa60 con 0x7f0a1815f650 2026-02-24T18:35:37.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 1 -- 10.20.193.18:0/2760464922 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0802fe30 con 0x7f0a1815f650 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 4 set_mon_vals no callback set 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a0f7fe640 10 monclient: dump: 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 10 monclient: shutdown 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/2760464922 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a1815f650 msgr2=0x7f0a1815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/2760464922 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a1815f650 0x7f0a1815fa30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a180ce0a0 tx=0x7f0a0802ed50 comp rx=0 tx=0).stop 2026-02-24T18:35:37.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/2760464922 shutdown_connections 2026-02-24T18:35:37.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/2760464922 >> v2:10.20.193.110:3300/0 conn(0x7f0a180ceca0 0x7f0a180d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/2760464922 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a1815f650 0x7f0a1815fa30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/2760464922 >> 10.20.193.18:0/2760464922 conn(0x7f0a18072fc0 msgr2=0x7f0a180733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/2760464922 shutdown_connections 2026-02-24T18:35:37.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.140+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/2760464922 wait complete. 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient: monmap: 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.145 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 librados: starting msgr at 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 librados: starting objecter 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 Processor -- start 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 -- start start 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 librados: setting wanted keys 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 librados: calling monclient init 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient: init 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0a180ceca0 0x7f0a181f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-b con 0x7f0a180ceca0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-a con 0x7f0a180d1ea0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0a1815f650 0x7f0a181f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): picked mon.noname-c con 0x7f0a1815f650 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180cf0f0 con 0x7f0a180d1ea0 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180c95e0 con 0x7f0a180ceca0 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181f4af0 con 0x7f0a1815f650 2026-02-24T18:35:37.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.141+0000 7f0a1ec1a640 10 monclient(hunting): authenticate will time out at 1480.822872s 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:41978/0 (socket says 10.20.193.18:41978) 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 1 -- 10.20.193.18:0/3283677090 learned_addr learned my addr 10.20.193.18:0/3283677090 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): get_auth_request con 0x7f0a180d1ea0 auth_method 0 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.142+0000 7f0a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 10 monclient(hunting): handle_auth_done global_id 4602 payload 824 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 -- 10.20.193.18:0/3283677090 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0a1815f650 msgr2=0x7f0a181f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0a1815f650 0x7f0a181f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 -- 10.20.193.18:0/3283677090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0a180ceca0 msgr2=0x7f0a181f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0a180ceca0 0x7f0a181f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 monclient: found mon.noname-a 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.143+0000 7f0a0ffff640 1 -- 10.20.193.18:0/3283677090 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a181ea320 con 0x7f0a180d1ea0 2026-02-24T18:35:37.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.144+0000 7f0a0ffff640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b530 tx=0x7f0a0000ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.144+0000 7f0a0d7fa640 10 client.?.objecter ms_handle_connect 0x7f0a180d1ea0 2026-02-24T18:35:37.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.144+0000 7f0a0d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.6 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00013020 con 0x7f0a180d1ea0 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: dump: 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a00004480 con 0x7f0a180d1ea0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00002920 con 0x7f0a180d1ea0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0d7fa640 10 monclient: dump: 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a0effd640 4 set_mon_vals no callback set 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 5 monclient: authenticate success, global_id 4602 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 10 monclient: _renew_subs 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181f5140 con 0x7f0a180d1ea0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 10 client.4602.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 10 monclient: _renew_subs 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181f5700 con 0x7f0a180d1ea0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.146+0000 7f0a1ec1a640 1 librados: init done 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.147+0000 7f0a1ec1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.147+0000 7f0a1ec1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.147+0000 7f0a1ec1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.147+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a18074ea0 con 0x7f0a180d1ea0 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.157+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0a00002ac0 con 0x7f0a180d1ea0 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7f0a0d7fa640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 0x7f09e807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 10 monclient: monmap: 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.159128+0000 2026-02-24T18:35:37.159 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.159128+0000 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7f0a1c98f640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 0x7f09e807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7f0a1c98f640 10 monclient: get_auth_request con 0x7f09e8078e00 auth_method 0 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 1 Processor -- start 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 1 -- start start 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7fb8eea58640 10 monclient: init 2026-02-24T18:35:37.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7f0a0d7fa640 10 client.4602.objecter ms_handle_connect 0x7f09e8078e00 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.158+0000 7f0a1c98f640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 0x7f09e807b2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a08039000 tx=0x7f0a08031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient: _add_conns ranks=[5,3,2,1,4,0] 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fb8e806ff20 0x7fb8e8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-c con 0x7fb8e806ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb8e80c99a0 0x7fb8e8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-a con 0x7fb8e80c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb8e815a410 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e815a860 con 0x7fb8e815a410 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e8159a40 con 0x7fb8e80c99a0 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e80c7e90 con 0x7fb8e806ff20 2026-02-24T18:35:37.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.159+0000 7fb8eea58640 10 monclient(hunting): authenticate will time out at 1480.840293s 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.160+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0a0005de50 con 0x7f0a180d1ea0 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.160+0000 7f0a0d7fa640 10 client.4602.objecter ms_dispatch2 0x7f0a18000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.160+0000 7f0a0d7fa640 3 client.4602.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.160+0000 7f0a0d7fa640 3 client.4602.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.160+0000 7f0a0d7fa640 20 client.4602.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.175+0000 7fa92bfff640 1 -- 10.20.193.18:0/3113133982 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa944061790 con 0x7fa9540bc8f0 2026-02-24T18:35:37.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.175+0000 7fa92bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:37.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.175+0000 7fa92bfff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.177 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:37.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.177+0000 7fa9598f3640 10 librados: watch_flush enter 2026-02-24T18:35:37.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.177+0000 7fa9598f3640 10 librados: watch_flush exit 2026-02-24T18:35:37.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.177+0000 7fa9598f3640 20 client.4549.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.177+0000 7fa9598f3640 10 client.4549.objecter successfully canceled tick 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 msgr2=0x7fa92407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 0x7fa92407b2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa954064240 tx=0x7fa948009040 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 10 monclient: shutdown 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 msgr2=0x7fa9540bcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9540c55f0 tx=0x7fa94400f5a0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 shutdown_connections 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa924078e30 0x7fa92407b2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9540bc8f0 0x7fa9540bcd70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa954157790 0x7fa9541ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 --2- 10.20.193.18:0/3113133982 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa940004900 0x7fa9541ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 >> 10.20.193.18:0/3113133982 conn(0x7fa9540cd780 msgr2=0x7fa9541e9940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 shutdown_connections 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 -- 10.20.193.18:0/3113133982 wait complete. 2026-02-24T18:35:37.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.178+0000 7fa9598f3640 1 librados: shutdown 2026-02-24T18:35:37.185 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-24T18:35:37.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.5 2026-02-24T18:35:37.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.209+0000 7ffbb4ecd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:37.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.209+0000 7ffbb4ecd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:37.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.209+0000 7ffbb4ecd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.209+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ffbb015a070 con 0x7ffbb006ff20 2026-02-24T18:35:37.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.211+0000 7ffb96ffd640 1 -- 10.20.193.18:0/2561662353 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffba4061b90 con 0x7ffbb006ff20 2026-02-24T18:35:37.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.211+0000 7ffb96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:37.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.211+0000 7ffb96ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.213 INFO:teuthology.orchestra.run.trial018.stdout:98784247810 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 10 librados: watch_flush enter 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 10 librados: watch_flush exit 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 20 client.4555.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 10 client.4555.objecter successfully canceled tick 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 msgr2=0x7ffb7c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 0x7ffb7c07b2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 10 monclient: shutdown 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 msgr2=0x7ffbb0147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b7e0 tx=0x7ffba400bcb0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 shutdown_connections 2026-02-24T18:35:37.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ffb7c078e00 0x7ffb7c07b2c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.213+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ffbb015a5e0 0x7ffbb01f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ffbb00c99a0 0x7ffbb00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 --2- 10.20.193.18:0/2561662353 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ffbb006ff20 0x7ffbb0147320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 >> 10.20.193.18:0/2561662353 conn(0x7ffbb00cd780 msgr2=0x7ffbb00c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 shutdown_connections 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 -- 10.20.193.18:0/2561662353 wait complete. 2026-02-24T18:35:37.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.214+0000 7ffbb4ecd640 1 librados: shutdown 2026-02-24T18:35:37.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.3 2026-02-24T18:35:37.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.229+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f67b80969b0 con 0x7f67cc1527f0 2026-02-24T18:35:37.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.230+0000 7f67c8ff9640 10 client.4550.objecter ms_dispatch2 0x7f67cc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.230+0000 7f67c8ff9640 3 client.4550.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.230+0000 7f67c8ff9640 3 client.4550.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.230+0000 7f67c8ff9640 20 client.4550.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.236+0000 7fb8ecfce640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fb8e815a410 0x7fb8e814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58376/0 (socket says 10.20.193.18:58376) 2026-02-24T18:35:37.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.236+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f67b80619a0 con 0x7f67cc1527f0 2026-02-24T18:35:37.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.236+0000 7f67c8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.236+0000 7fb8ecfce640 1 -- 10.20.193.18:0/3017573854 learned_addr learned my addr 10.20.193.18:0/3017573854 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:37.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.236+0000 7f67c8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.240+0000 7fb8e7fff640 1 --2- 10.20.193.18:0/3017573854 >> v2:10.20.193.110:3300/0 conn(0x7fb8e806ff20 0x7fb8e8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.240+0000 7fb8e7fff640 10 monclient(hunting): get_auth_request con 0x7fb8e806ff20 auth_method 0 2026-02-24T18:35:37.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.240+0000 7fb8e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.240+0000 7fb8e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.240+0000 7fb8e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.241+0000 7fb8e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.241+0000 7fb8e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.241+0000 7fb8e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 10 monclient(hunting): handle_auth_done global_id 4553 payload 274 2026-02-24T18:35:37.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 1 -- 10.20.193.18:0/3017573854 >> v2:10.20.193.18:3300/0 conn(0x7fb8e80c99a0 msgr2=0x7fb8e8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 1 --2- 10.20.193.18:0/3017573854 >> v2:10.20.193.18:3300/0 conn(0x7fb8e80c99a0 0x7fb8e8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 1 -- 10.20.193.18:0/3017573854 >> v1:10.20.193.110:6789/0 conn(0x7fb8e815a410 legacy=0x7fb8e814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.242+0000 7fb8e7fff640 1 -- 10.20.193.18:0/3017573854 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e80c8170 con 0x7fb8e806ff20 2026-02-24T18:35:37.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.243+0000 7fb8e7fff640 1 --2- 10.20.193.18:0/3017573854 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e806ff20 0x7fb8e8070300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098a0 tx=0x7fb8d402ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832a53bf931985f1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 1 -- 10.20.193.18:0/3017573854 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d403c070 con 0x7fb8e806ff20 2026-02-24T18:35:37.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: dump: 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 1 -- 10.20.193.18:0/3017573854 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d402fae0 con 0x7fb8e806ff20 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 1 -- 10.20.193.18:0/3017573854 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4037440 con 0x7fb8e806ff20 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8e6ffd640 10 monclient: dump: 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.248 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.249 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 5 monclient: authenticate success, global_id 4553 2026-02-24T18:35:37.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 4 set_mon_vals no callback set 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 10 monclient: shutdown 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 1 -- 10.20.193.18:0/3017573854 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e806ff20 msgr2=0x7fb8e8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.245+0000 7fb8eea58640 1 --2- 10.20.193.18:0/3017573854 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e806ff20 0x7fb8e8070300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098a0 tx=0x7fb8d402ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 -- 10.20.193.18:0/3017573854 shutdown_connections 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 --2- 10.20.193.18:0/3017573854 >> v2:10.20.193.18:3300/0 conn(0x7fb8e80c99a0 0x7fb8e8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 --2- 10.20.193.18:0/3017573854 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e806ff20 0x7fb8e8070300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 -- 10.20.193.18:0/3017573854 >> 10.20.193.18:0/3017573854 conn(0x7fb8e80cd780 msgr2=0x7fb8e80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 -- 10.20.193.18:0/3017573854 shutdown_connections 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 -- 10.20.193.18:0/3017573854 wait complete. 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 10 monclient: monmap: 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 librados: starting msgr at 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 librados: starting objecter 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 Processor -- start 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 -- start start 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 librados: setting wanted keys 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 1 librados: calling monclient init 2026-02-24T18:35:37.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.246+0000 7fb8eea58640 10 monclient: init 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb8e806ff20 0x7fb8e81e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-a con 0x7fb8e806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-c con 0x7fb8e80c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb8e815a410 0x7fb8e81f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): picked mon.noname-b con 0x7fb8e815a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80c8a10 con 0x7fb8e806ff20 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80c8730 con 0x7fb8e815a410 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8150520 con 0x7fb8e80c99a0 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8eea58640 10 monclient(hunting): authenticate will time out at 1480.928289s 2026-02-24T18:35:37.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44670/0 (socket says 10.20.193.18:44670) 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 1 -- 10.20.193.18:0/2917349375 learned_addr learned my addr 10.20.193.18:0/2917349375 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 10 monclient(hunting): get_auth_request con 0x7fb8e80c99a0 auth_method 0 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.247+0000 7fb8ecfce640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb8e815a410 0x7fb8e81f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): get_auth_request con 0x7fb8e815a410 auth_method 0 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.248+0000 7fb8ecfce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 10 monclient(hunting): handle_auth_done global_id 4556 payload 824 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 -- 10.20.193.18:0/2917349375 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb8e815a410 msgr2=0x7fb8e81f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb8e815a410 0x7fb8e81f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 -- 10.20.193.18:0/2917349375 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb8e806ff20 msgr2=0x7fb8e81e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb8e806ff20 0x7fb8e81e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.249+0000 7fb8e77fe640 1 -- 10.20.193.18:0/2917349375 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e81e5810 con 0x7fb8e80c99a0 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.250+0000 7fb8e77fe640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800b780 tx=0x7fb8d800bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.250+0000 7fb8e4ff9640 10 client.?.objecter ms_handle_connect 0x7fb8e80c99a0 2026-02-24T18:35:37.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.250+0000 7fb8e4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004480 con 0x7fb8e80c99a0 2026-02-24T18:35:37.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: dump: 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d8010070 con 0x7fb8e80c99a0 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8002920 con 0x7fb8e80c99a0 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e67fc640 4 set_mon_vals no callback set 2026-02-24T18:35:37.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8e4ff9640 10 monclient: dump: 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 5 monclient: authenticate success, global_id 4556 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 10 monclient: _renew_subs 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81e5af0 con 0x7fb8e80c99a0 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 10 client.4556.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 10 monclient: _renew_subs 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81e60b0 con 0x7fb8e80c99a0 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.252+0000 7fb8eea58640 1 librados: init done 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fb8eea58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fb8eea58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fb8eea58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e81e66c0 con 0x7fb8e80c99a0 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fe98329f640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fe98329f640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.253+0000 7fe98329f640 10 monclient: monmap: 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.254734+0000 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.254734+0000 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.254+0000 7fe98329f640 1 Processor -- start 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.254+0000 7fe98329f640 1 -- start start 2026-02-24T18:35:37.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.254+0000 7fe98329f640 10 monclient: init 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient: _add_conns ranks=[4,2,3,1,5,0] 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe97c1527f0 0x7fe97c073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-b con 0x7fe97c1527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe97c15d2c0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe97c15d730 0x7fe97c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-a con 0x7fe97c15d730 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c073c20 con 0x7fe97c15d2c0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c074b10 con 0x7fe97c15d730 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c073f00 con 0x7fe97c1527f0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe98329f640 10 monclient(hunting): authenticate will time out at 1480.936281s 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe97c1527f0 0x7fe97c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe97c1527f0 0x7fe97c073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53730/0 (socket says 10.20.193.18:53730) 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 1 -- 10.20.193.18:0/4223117644 learned_addr learned my addr 10.20.193.18:0/4223117644 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 10 monclient(hunting): get_auth_request con 0x7fe97c1527f0 auth_method 0 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.255+0000 7fe981014640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.256+0000 7fe981014640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.256+0000 7fe981014640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.256+0000 7fe981014640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 10 monclient(hunting): handle_auth_done global_id 4561 payload 274 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 1 -- 10.20.193.18:0/4223117644 >> v2:10.20.193.18:3300/0 conn(0x7fe97c15d730 msgr2=0x7fe97c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 1 --2- 10.20.193.18:0/4223117644 >> v2:10.20.193.18:3300/0 conn(0x7fe97c15d730 0x7fe97c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 1 -- 10.20.193.18:0/4223117644 >> v1:10.20.193.110:6789/0 conn(0x7fe97c15d2c0 legacy=0x7fe97c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.257+0000 7fe981014640 1 -- 10.20.193.18:0/4223117644 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c0741e0 con 0x7fe97c1527f0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.258+0000 7fe981014640 1 --2- 10.20.193.18:0/4223117644 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c1527f0 0x7fe97c073530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe97c06e9a0 tx=0x7fe96c02eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f977f41cfe9b3969 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 1 -- 10.20.193.18:0/4223117644 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c03c070 con 0x7fe97c1527f0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: dump: 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.264 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe98329f640 5 monclient: authenticate success, global_id 4561 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe98329f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 1 -- 10.20.193.18:0/4223117644 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c02fb40 con 0x7fe97c1527f0 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 1 -- 10.20.193.18:0/4223117644 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037440 con 0x7fe97c1527f0 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe96bfff640 10 monclient: dump: 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.265 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe98329f640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.259+0000 7fe98329f640 4 set_mon_vals no callback set 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 10 monclient: shutdown 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 -- 10.20.193.18:0/4223117644 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c1527f0 msgr2=0x7fe97c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 --2- 10.20.193.18:0/4223117644 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c1527f0 0x7fe97c073530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe97c06e9a0 tx=0x7fe96c02eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 -- 10.20.193.18:0/4223117644 shutdown_connections 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 --2- 10.20.193.18:0/4223117644 >> v2:10.20.193.18:3300/0 conn(0x7fe97c15d730 0x7fe97c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 --2- 10.20.193.18:0/4223117644 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c1527f0 0x7fe97c073530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 -- 10.20.193.18:0/4223117644 >> 10.20.193.18:0/4223117644 conn(0x7fe97c0ce730 msgr2=0x7fe97c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 -- 10.20.193.18:0/4223117644 shutdown_connections 2026-02-24T18:35:37.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.260+0000 7fe98329f640 1 -- 10.20.193.18:0/4223117644 wait complete. 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient: monmap: 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 librados: starting msgr at 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 librados: starting objecter 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 Processor -- start 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 -- start start 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 librados: setting wanted keys 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 librados: calling monclient init 2026-02-24T18:35:37.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient: init 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe97c1527f0 0x7fe97c1f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-a con 0x7fe97c1527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe97c15d2c0 0x7fe97c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-c con 0x7fe97c15d2c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): picked mon.noname-b con 0x7fe97c15d730 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c074fa0 con 0x7fe97c1527f0 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c0747a0 con 0x7fe97c15d730 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c075120 con 0x7fe97c15d2c0 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.261+0000 7fe98329f640 10 monclient(hunting): authenticate will time out at 1480.942804s 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53732/0 (socket says 10.20.193.18:53732) 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 1 -- 10.20.193.18:0/85315979 learned_addr learned my addr 10.20.193.18:0/85315979 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): get_auth_request con 0x7fe97c15d730 auth_method 0 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.262+0000 7fe981815640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 10 monclient(hunting): handle_auth_done global_id 4564 payload 824 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 -- 10.20.193.18:0/85315979 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe97c15d2c0 msgr2=0x7fe97c0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe97c15d2c0 0x7fe97c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 -- 10.20.193.18:0/85315979 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe97c1527f0 msgr2=0x7fe97c1f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe97c1527f0 0x7fe97c1f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.263+0000 7fe981815640 1 -- 10.20.193.18:0/85315979 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c0c7550 con 0x7fe97c15d730 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb8d8002ac0 con 0x7fb8e80c99a0 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fb8e4ff9640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 0x7fb8b807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fe981815640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe97800b9d0 tx=0x7fe97800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fe969ffb640 10 client.?.objecter ms_handle_connect 0x7fe97c15d730 2026-02-24T18:35:37.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fe969ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fb8e7fff640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 0x7fb8b807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.265+0000 7fb8e7fff640 10 monclient: get_auth_request con 0x7fb8b8078e30 auth_method 0 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e7fff640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 0x7fb8b807b2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4039000 tx=0x7fb8d4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 10 client.4556.objecter ms_handle_connect 0x7fb8b8078e30 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fb8d805d5c0 con 0x7fb8e80c99a0 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 10 client.4556.objecter ms_dispatch2 0x7fb8e8000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 3 client.4556.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 3 client.4556.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.266+0000 7fb8e4ff9640 20 client.4556.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97800c7c0 con 0x7fe97c15d730 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: dump: 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe978010070 con 0x7fe97c15d730 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97800cbc0 con 0x7fe97c15d730 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe969ffb640 10 monclient: dump: 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe96b7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 5 monclient: authenticate success, global_id 4564 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 10 monclient: _renew_subs 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c0c7830 con 0x7fe97c15d730 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 10 client.4564.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 10 monclient: _renew_subs 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c0c7df0 con 0x7fe97c15d730 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.267+0000 7fe98329f640 1 librados: init done 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.268+0000 7fe98329f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.268+0000 7fe98329f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.268+0000 7fe98329f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.268+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c152bd0 con 0x7fe97c15d730 2026-02-24T18:35:37.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.271+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0a0001d020 con 0x7f0a180d1ea0 2026-02-24T18:35:37.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.271+0000 7f0a0d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.271+0000 7f0a0d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.279+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe978004070 con 0x7fe97c15d730 2026-02-24T18:35:37.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.279+0000 7fe969ffb640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 0x7fe94c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.280+0000 7fe981014640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 0x7fe94c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.280+0000 7fe981014640 10 monclient: get_auth_request con 0x7fe94c078e50 auth_method 0 2026-02-24T18:35:37.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.280+0000 7fe981014640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 0x7fe94c07b310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe97c08f510 tx=0x7fe96c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.280+0000 7fe969ffb640 10 client.4564.objecter ms_handle_connect 0x7fe94c078e50 2026-02-24T18:35:37.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.281+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fe97800cd60 con 0x7fe97c15d730 2026-02-24T18:35:37.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.281+0000 7fe969ffb640 10 client.4564.objecter ms_dispatch2 0x7fe97c000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.281+0000 7fe969ffb640 3 client.4564.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.281+0000 7fe969ffb640 3 client.4564.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.281+0000 7fe969ffb640 20 client.4564.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.325+0000 7f67d2511640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:37.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.325+0000 7f67d2511640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:37.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.325+0000 7f67d2511640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.325+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f67cc152bd0 con 0x7f67cc1527f0 2026-02-24T18:35:37.349 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-24T18:35:37.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.363+0000 7f0a1ec1a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:37.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.363+0000 7f0a1ec1a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:37.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.363+0000 7f0a1ec1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.363+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f0a18150970 con 0x7f0a180d1ea0 2026-02-24T18:35:37.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7f0a0d7fa640 1 -- 10.20.193.18:0/3283677090 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f0a00062510 con 0x7f0a180d1ea0 2026-02-24T18:35:37.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7f0a0d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-24T18:35:37.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7f0a0d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.368 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:37.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb8d8024080 con 0x7fb8e80c99a0 2026-02-24T18:35:37.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7fb8e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.366+0000 7fb8e4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 10 librados: watch_flush enter 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 10 librados: watch_flush exit 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 20 client.4602.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 10 client.4602.objecter successfully canceled tick 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 msgr2=0x7f09e807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 0x7f09e807b2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a08039000 tx=0x7f0a08031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 10 monclient: shutdown 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 msgr2=0x7f0a181556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b530 tx=0x7f0a0000ba00 comp rx=0 tx=0).stop 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 shutdown_connections 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f09e8078e00 0x7f09e807b2c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0a1815f650 0x7f0a181f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0a180d1ea0 0x7f0a181556c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 --2- 10.20.193.18:0/3283677090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0a180ceca0 0x7f0a181f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 >> 10.20.193.18:0/3283677090 conn(0x7f0a18072fc0 msgr2=0x7f0a18150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 shutdown_connections 2026-02-24T18:35:37.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 -- 10.20.193.18:0/3283677090 wait complete. 2026-02-24T18:35:37.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.368+0000 7f0a1ec1a640 1 librados: shutdown 2026-02-24T18:35:37.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.371+0000 7f67c8ff9640 1 -- 10.20.193.18:0/2297947437 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f67b80612a0 con 0x7f67cc1527f0 2026-02-24T18:35:37.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.371+0000 7f67c8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:37.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.371+0000 7f67c8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.373 INFO:teuthology.orchestra.run.trial018.stdout:94489280515 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 10 librados: watch_flush enter 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 10 librados: watch_flush exit 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 20 client.4550.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 10 client.4550.objecter successfully canceled tick 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 msgr2=0x7f679c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 0x7f679c07b2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67cc06cdf0 tx=0x7f67bc031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 10 monclient: shutdown 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 msgr2=0x7f67cc0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b7b0 tx=0x7f67b800bc80 comp rx=0 tx=0).stop 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 shutdown_connections 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f679c078e00 0x7f679c07b2c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f67cc15d2c0 0x7f67cc1f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f67cc15d730 0x7f67cc1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 --2- 10.20.193.18:0/2297947437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67cc1527f0 0x7f67cc0c82c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 >> 10.20.193.18:0/2297947437 conn(0x7f67cc0ce730 msgr2=0x7f67cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 shutdown_connections 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 -- 10.20.193.18:0/2297947437 wait complete. 2026-02-24T18:35:37.377 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.374+0000 7f67d2511640 1 librados: shutdown 2026-02-24T18:35:37.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.14 2026-02-24T18:35:37.380 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.381 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-24T18:35:37.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.381+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe9780622e0 con 0x7fe97c15d730 2026-02-24T18:35:37.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.381+0000 7fe969ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.381+0000 7fe969ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.11 2026-02-24T18:35:37.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 10 monclient: monmap: 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.423343+0000 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.423343+0000 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.425 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.426 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.426 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 1 Processor -- start 2026-02-24T18:35:37.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 1 -- start start 2026-02-24T18:35:37.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.422+0000 7f63b6d7e640 10 monclient: init 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient: _add_conns ranks=[2,0,1,5,4,3] 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f63b006ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f63b00c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f63b015a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:37.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b00c7e90 con 0x7f63b00c99a0 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b00c8170 con 0x7f63b015a5e0 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b00c8450 con 0x7f63b006ff20 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63b6d7e640 10 monclient(hunting): authenticate will time out at 1481.104542s 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63a7fff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f63b00c99a0 0x7f63b014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58734/0 (socket says 10.20.193.18:58734) 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.423+0000 7f63a7fff640 1 -- 10.20.193.18:0/2292277745 learned_addr learned my addr 10.20.193.18:0/2292277745 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903434922 0 0) 0x7f63b00c7e90 con 0x7f63b00c99a0 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): my global_id is 4605 2026-02-24T18:35:37.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f639c003620 con 0x7f63b00c99a0 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1253452504 0 0) 0x7f63b00c8450 con 0x7f63b006ff20 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): my global_id is 4559 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63b00c7e90 con 0x7f63b006ff20 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392350025 0 0) 0x7f63b00c8170 con 0x7f63b015a5e0 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _negotiate 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 10 monclient(hunting): my global_id is 4567 2026-02-24T18:35:37.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.425+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63b00c8450 con 0x7f63b015a5e0 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2407290104 0 0) 0x7f63b00c7e90 con 0x7f63b006ff20 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 >> v1:10.20.193.32:6789/0 conn(0x7f63b015a5e0 legacy=0x7f63b00c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 >> v1:10.20.193.18:6789/0 conn(0x7f63b00c99a0 legacy=0x7f63b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b00c8730 con 0x7f63b006ff20 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f63a0002ee0 con 0x7f63b006ff20 2026-02-24T18:35:37.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63b6d7e640 5 monclient: authenticate success, global_id 4559 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63b6d7e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: dump: 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.435 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 >> v1:10.20.193.110:6789/0 conn(0x7f63b006ff20 legacy=0x7f63b0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f639c005040 0x7f639c0038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): picked mon.a con 0x7f639c005040 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c003de0 0x7f639c004260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): picked mon.c con 0x7f639c003de0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 0x7f639c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): picked mon.b con 0x7f639c009ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f639c00c570 con 0x7f639c005040 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f639c00c6f0 con 0x7f639c009ab0 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f639c00c8f0 con 0x7f639c003de0 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63a77fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.426+0000 7f63b6d7e640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.427+0000 7f63b52f4640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 0x7f639c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.427+0000 7f63b52f4640 10 monclient(hunting): get_auth_request con 0x7f639c009ab0 auth_method 0 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.427+0000 7f63b52f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.427+0000 7f63b52f4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.427+0000 7f63b52f4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c003de0 0x7f639c004260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): get_auth_request con 0x7f639c003de0 auth_method 0 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.428+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:37.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 10 monclient(hunting): handle_auth_done global_id 4559 payload 293 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 -- 10.20.193.18:0/2292277745 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c003de0 msgr2=0x7f639c004260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c003de0 0x7f639c004260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 -- 10.20.193.18:0/2292277745 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f639c005040 msgr2=0x7f639c0038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f639c005040 0x7f639c0038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 monclient: found mon.b 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 -- 10.20.193.18:0/2292277745 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b00c7e90 con 0x7f639c009ab0 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.429+0000 7f63b52f4640 1 -- 10.20.193.18:0/2292277745 reap_dead start 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.430+0000 7f63b52f4640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 0x7f639c00bea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c320 tx=0x7f63ac02f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1093236e15a9da73 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac003860 con 0x7f639c009ab0 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: dump: 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.446 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac003c70 con 0x7f639c009ab0 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 1 -- 10.20.193.18:0/2292277745 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac005310 con 0x7f639c009ab0 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 4 set_mon_vals no callback set 2026-02-24T18:35:37.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63a77fe640 10 monclient: dump: 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 10 monclient: shutdown 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 -- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 msgr2=0x7f639c00bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 0x7f639c00bea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c320 tx=0x7f63ac02f610 comp rx=0 tx=0).stop 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 -- 10.20.193.18:0/2292277745 shutdown_connections 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/2292277745 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f639c009ab0 0x7f639c00bea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 -- 10.20.193.18:0/2292277745 >> 10.20.193.18:0/2292277745 conn(0x7f63b00cd780 msgr2=0x7f63b00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 -- 10.20.193.18:0/2292277745 shutdown_connections 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.432+0000 7f63b6d7e640 1 -- 10.20.193.18:0/2292277745 wait complete. 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient: monmap: 2026-02-24T18:35:37.449 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 librados: starting msgr at 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 librados: starting objecter 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 Processor -- start 2026-02-24T18:35:37.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 -- start start 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 librados: setting wanted keys 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 librados: calling monclient init 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient: init 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c009ab0 0x7f63b0149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-c con 0x7f639c009ab0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f63b00c3ad0 0x7f63b00c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-b con 0x7f63b00c3ad0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): picked mon.noname-a con 0x7f63b00c2fd0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0070300 con 0x7f63b00c2fd0 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0159a40 con 0x7f63b00c3ad0 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0070480 con 0x7f639c009ab0 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.433+0000 7f63b6d7e640 10 monclient(hunting): authenticate will time out at 1481.114786s 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f63b00c3ad0 0x7f63b00c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42036/0 (socket says 10.20.193.18:42036) 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 1 -- 10.20.193.18:0/3730499904 learned_addr learned my addr 10.20.193.18:0/3730499904 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): get_auth_request con 0x7f63b00c2fd0 auth_method 0 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): get_auth_request con 0x7f63b00c3ad0 auth_method 0 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c009ab0 0x7f63b0149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): get_auth_request con 0x7f639c009ab0 auth_method 0 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b52f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.434+0000 7f63b4af3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 10 monclient(hunting): handle_auth_done global_id 4608 payload 824 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 -- 10.20.193.18:0/3730499904 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c009ab0 msgr2=0x7f63b0149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c009ab0 0x7f63b0149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 -- 10.20.193.18:0/3730499904 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f63b00c3ad0 msgr2=0x7f63b00c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f63b00c3ad0 0x7f63b00c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 monclient: found mon.noname-a 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.435+0000 7f63b52f4640 1 -- 10.20.193.18:0/3730499904 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b0149030 con 0x7f63b00c2fd0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.436+0000 7f63b52f4640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c450 tx=0x7f63ac002e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.436+0000 7f63a57fa640 10 client.?.objecter ms_handle_connect 0x7f63b00c2fd0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.436+0000 7f63a57fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac003860 con 0x7f63b00c2fd0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:37.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: dump: 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac036610 con 0x7f63b00c2fd0 2026-02-24T18:35:37.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0369c0 con 0x7f63b00c2fd0 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 5 monclient: authenticate success, global_id 4608 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a6ffd640 4 set_mon_vals no callback set 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 10 monclient: _renew_subs 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01f4760 con 0x7f63b00c2fd0 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63a57fa640 10 monclient: dump: 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.457 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 10 client.4608.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 10 monclient: _renew_subs 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01f7190 con 0x7f63b00c2fd0 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.438+0000 7f63b6d7e640 1 librados: init done 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.439+0000 7f63b6d7e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.439+0000 7f63b6d7e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.439+0000 7f63b6d7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.439+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b0159ed0 con 0x7f63b00c2fd0 2026-02-24T18:35:37.458 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.449+0000 7f8f3795c640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.449+0000 7f8f3795c640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.449+0000 7f8f3795c640 10 monclient: monmap: 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.450538+0000 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.450538+0000 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.460 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 Processor -- start 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 -- start start 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient: init 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient: _add_conns ranks=[0,3,4,1,5,2] 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8f30155c70 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8f3015d9d0 0x7f8f3015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-a con 0x7f8f3015d9d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:37.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f8f300ccb30 0x7f8f3015f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-b con 0x7f8f300ccb30 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f30072e90 con 0x7f8f30155c70 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3006f440 con 0x7f8f3015d9d0 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8f300cbcb0 con 0x7f8f300ccb30 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f8f3795c640 10 monclient(hunting): authenticate will time out at 1481.131693s 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.450+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f63ac036b60 con 0x7f63b00c2fd0 2026-02-24T18:35:37.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f8f300ccb30 0x7f8f3015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f63a57fa640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 0x7f638007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f8f300ccb30 0x7f8f3015f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53766/0 (socket says 10.20.193.18:53766) 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 1 -- 10.20.193.18:0/1758510057 learned_addr learned my addr 10.20.193.18:0/1758510057 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): get_auth_request con 0x7f8f300ccb30 auth_method 0 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f63b4af3640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 0x7f638007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f63b4af3640 10 monclient: get_auth_request con 0x7f6380078e50 auth_method 0 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f8f35ed2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f63b4af3640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 0x7f638007b310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63a0002820 tx=0x7f63a00027b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.451+0000 7f63a57fa640 10 client.4608.objecter ms_handle_connect 0x7f6380078e50 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 10 monclient(hunting): handle_auth_done global_id 4573 payload 274 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 1 -- 10.20.193.18:0/1758510057 >> v2:10.20.193.18:3300/0 conn(0x7f8f3015d9d0 msgr2=0x7f8f3015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 1 --2- 10.20.193.18:0/1758510057 >> v2:10.20.193.18:3300/0 conn(0x7f8f3015d9d0 0x7f8f3015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 1 -- 10.20.193.18:0/1758510057 >> v1:10.20.193.18:6789/0 conn(0x7f8f30155c70 legacy=0x7f8f3015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.452+0000 7f8f35ed2640 1 -- 10.20.193.18:0/1758510057 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3006f690 con 0x7f8f300ccb30 2026-02-24T18:35:37.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.453+0000 7f8f35ed2640 1 --2- 10.20.193.18:0/1758510057 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f300ccb30 0x7f8f3015f900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f30069a40 tx=0x7f8f2c02f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42dd7bbeaa90ae3 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f63ac082cc0 con 0x7f63b00c2fd0 2026-02-24T18:35:37.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f63a57fa640 10 client.4608.objecter ms_dispatch2 0x7f63b0000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f63a57fa640 3 client.4608.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f63a57fa640 3 client.4608.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f63a57fa640 20 client.4608.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 1 -- 10.20.193.18:0/1758510057 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c02fe00 con 0x7f8f300ccb30 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: dump: 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 1 -- 10.20.193.18:0/1758510057 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0373f0 con 0x7f8f300ccb30 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f3795c640 5 monclient: authenticate success, global_id 4573 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f3795c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 1 -- 10.20.193.18:0/1758510057 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0377c0 con 0x7f8f300ccb30 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f3795c640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f3795c640 4 set_mon_vals no callback set 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.454+0000 7f8f1ffff640 10 monclient: dump: 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 10 monclient: shutdown 2026-02-24T18:35:37.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- 10.20.193.18:0/1758510057 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f300ccb30 msgr2=0x7f8f3015f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 --2- 10.20.193.18:0/1758510057 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f300ccb30 0x7f8f3015f900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f30069a40 tx=0x7f8f2c02f3a0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- 10.20.193.18:0/1758510057 shutdown_connections 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 --2- 10.20.193.18:0/1758510057 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f300ccb30 0x7f8f3015f900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 --2- 10.20.193.18:0/1758510057 >> v2:10.20.193.18:3300/0 conn(0x7f8f3015d9d0 0x7f8f3015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- 10.20.193.18:0/1758510057 >> 10.20.193.18:0/1758510057 conn(0x7f8f300cc010 msgr2=0x7f8f300cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- 10.20.193.18:0/1758510057 shutdown_connections 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- 10.20.193.18:0/1758510057 wait complete. 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 10 monclient: monmap: 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.470 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 librados: starting msgr at 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 librados: starting objecter 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 Processor -- start 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 -- start start 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 librados: setting wanted keys 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 1 librados: calling monclient init 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.455+0000 7f8f3795c640 10 monclient: init 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7fb8eea58640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7fb8eea58640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7fb8eea58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7fb8e8070300 con 0x7fb8e80c99a0 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8f300ccb30 0x7f8f3015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-c con 0x7f8f300ccb30 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-b con 0x7f8f30155c70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8f3015d9d0 0x7f8f3014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): picked mon.noname-a con 0x7f8f3015d9d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3006ff30 con 0x7f8f3015d9d0 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3006fc50 con 0x7f8f30155c70 2026-02-24T18:35:37.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30070210 con 0x7f8f300ccb30 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f3795c640 10 monclient(hunting): authenticate will time out at 1481.137381s 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53768/0 (socket says 10.20.193.18:53768) 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 1 -- 10.20.193.18:0/2069898843 learned_addr learned my addr 10.20.193.18:0/2069898843 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): get_auth_request con 0x7f8f30155c70 auth_method 0 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.456+0000 7f8f34ed0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 10 monclient(hunting): handle_auth_done global_id 4576 payload 824 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 -- 10.20.193.18:0/2069898843 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8f300ccb30 msgr2=0x7f8f3015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8f300ccb30 0x7f8f3015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 -- 10.20.193.18:0/2069898843 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8f3015d9d0 msgr2=0x7f8f3014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8f3015d9d0 0x7f8f3014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7f8f34ed0640 1 -- 10.20.193.18:0/2069898843 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f301ea1c0 con 0x7f8f30155c70 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7fb8e4ff9640 1 -- 10.20.193.18:0/2917349375 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fb8d8061c80 con 0x7fb8e80c99a0 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7fb8e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.458+0000 7fb8e4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.459+0000 7f8f34ed0640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b7b0 tx=0x7f8f2400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.459+0000 7f8f1dffb640 10 client.?.objecter ms_handle_connect 0x7f8f30155c70 2026-02-24T18:35:37.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.459+0000 7f8f1dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 10 librados: watch_flush enter 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 10 librados: watch_flush exit 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 20 client.4556.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 10 client.4556.objecter successfully canceled tick 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 msgr2=0x7fb8b807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 0x7fb8b807b2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4039000 tx=0x7fb8d4031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 10 monclient: shutdown 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 msgr2=0x7fb8e814ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800b780 tx=0x7fb8d800bc50 comp rx=0 tx=0).stop 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 shutdown_connections 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb8b8078e30 0x7fb8b807b2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb8e815a410 0x7fb8e81f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb8e80c99a0 0x7fb8e814ed40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 --2- 10.20.193.18:0/2917349375 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb8e806ff20 0x7fb8e81e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 >> 10.20.193.18:0/2917349375 conn(0x7fb8e80cd780 msgr2=0x7fb8e80c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 shutdown_connections 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 -- 10.20.193.18:0/2917349375 wait complete. 2026-02-24T18:35:37.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7fb8eea58640 1 librados: shutdown 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24004480 con 0x7f8f30155c70 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 10 monclient: dump: 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 5 monclient: authenticate success, global_id 4576 2026-02-24T18:35:37.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 10 monclient: _renew_subs 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f301e7a50 con 0x7f8f30155c70 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 10 client.4576.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 10 monclient: _renew_subs 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3015e320 con 0x7f8f30155c70 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.460+0000 7f8f3795c640 1 librados: init done 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f3795c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f3795c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f3795c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30074850 con 0x7f8f30155c70 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f24010070 con 0x7f8f30155c70 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240093b0 con 0x7f8f30155c70 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1f7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.461+0000 7f8f1dffb640 10 monclient: dump: 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.479 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.465+0000 7fe98329f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.465+0000 7fe98329f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.465+0000 7fe98329f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.465+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe97c0d0240 con 0x7fe97c15d730 2026-02-24T18:35:37.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.13 2026-02-24T18:35:37.481 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.481 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-24T18:35:37.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 10 monclient: monmap: 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.540109+0000 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.540109+0000 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.541 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 1 Processor -- start 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 1 -- start start 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.539+0000 7f6abbf68640 10 monclient: init 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient: _add_conns ranks=[5,0,1,4,2,3] 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6ab4155c70 0x7f6ab415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-c con 0x7f6ab4155c70 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ab40ccb30 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6ab4156050 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab415cc60 con 0x7f6ab40ccb30 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab4072e90 con 0x7f6ab4156050 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab406f440 con 0x7f6ab4155c70 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6abbf68640 10 monclient(hunting): authenticate will time out at 1481.221329s 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6ab4155c70 0x7f6ab415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6ab4155c70 0x7f6ab415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44726/0 (socket says 10.20.193.18:44726) 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 1 -- 10.20.193.18:0/1067501461 learned_addr learned my addr 10.20.193.18:0/1067501461 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 10 monclient(hunting): get_auth_request con 0x7f6ab4155c70 auth_method 0 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.540+0000 7f6ab9cdd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.541+0000 7f6ab9cdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.541+0000 7f6ab9cdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.541+0000 7f6ab9cdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 10 monclient(hunting): handle_auth_done global_id 4565 payload 274 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 1 -- 10.20.193.18:0/1067501461 >> v1:10.20.193.32:6789/0 conn(0x7f6ab4156050 legacy=0x7f6ab415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 1 -- 10.20.193.18:0/1067501461 >> v1:10.20.193.18:6789/0 conn(0x7f6ab40ccb30 legacy=0x7f6ab415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.542+0000 7f6ab9cdd640 1 -- 10.20.193.18:0/1067501461 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab40cbcb0 con 0x7f6ab4155c70 2026-02-24T18:35:37.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.543+0000 7f6ab9cdd640 1 --2- 10.20.193.18:0/1067501461 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4155c70 0x7f6ab415d2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009970 tx=0x7f6aa402eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea500092ab6ee454 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 1 -- 10.20.193.18:0/1067501461 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa403c070 con 0x7f6ab4155c70 2026-02-24T18:35:37.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: dump: 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6abbf68640 5 monclient: authenticate success, global_id 4565 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6abbf68640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 1 -- 10.20.193.18:0/1067501461 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa402fb60 con 0x7f6ab4155c70 2026-02-24T18:35:37.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 1 -- 10.20.193.18:0/1067501461 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4037440 con 0x7f6ab4155c70 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6ab8cdb640 10 monclient: dump: 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6abbf68640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.544+0000 7f6abbf68640 4 set_mon_vals no callback set 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 10 monclient: shutdown 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 -- 10.20.193.18:0/1067501461 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4155c70 msgr2=0x7f6ab415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1067501461 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4155c70 0x7f6ab415d2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009970 tx=0x7f6aa402eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 -- 10.20.193.18:0/1067501461 shutdown_connections 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1067501461 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4155c70 0x7f6ab415d2c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 -- 10.20.193.18:0/1067501461 >> 10.20.193.18:0/1067501461 conn(0x7f6ab40cc010 msgr2=0x7f6ab40cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 -- 10.20.193.18:0/1067501461 shutdown_connections 2026-02-24T18:35:37.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 -- 10.20.193.18:0/1067501461 wait complete. 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 10 monclient: monmap: 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 librados: starting msgr at 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.545+0000 7f6abbf68640 1 librados: starting objecter 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 Processor -- start 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 -- start start 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 librados: setting wanted keys 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 librados: calling monclient init 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient: init 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6ab40ccb30 0x7f6ab41e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-a con 0x7f6ab40ccb30 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6ab4155c70 0x7f6ab415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-b con 0x7f6ab4155c70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): picked mon.noname-c con 0x7f6ab4156050 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406fc50 con 0x7f6ab40ccb30 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406f970 con 0x7f6ab4155c70 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406ff30 con 0x7f6ab4156050 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.546+0000 7f6abbf68640 10 monclient(hunting): authenticate will time out at 1481.227673s 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44736/0 (socket says 10.20.193.18:44736) 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 1 -- 10.20.193.18:0/1201956293 learned_addr learned my addr 10.20.193.18:0/1201956293 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): get_auth_request con 0x7f6ab4156050 auth_method 0 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.547+0000 7f6aba4de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 10 monclient(hunting): handle_auth_done global_id 4568 payload 824 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 -- 10.20.193.18:0/1201956293 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6ab4155c70 msgr2=0x7f6ab415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6ab4155c70 0x7f6ab415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 -- 10.20.193.18:0/1201956293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6ab40ccb30 msgr2=0x7f6ab41e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6ab40ccb30 0x7f6ab41e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.548+0000 7f6aba4de640 1 -- 10.20.193.18:0/1201956293 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab40c7aa0 con 0x7f6ab4156050 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.549+0000 7f6aba4de640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b9c0 tx=0x7f6ab000be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.549+0000 7f6aa27fc640 10 client.?.objecter ms_handle_connect 0x7f6ab4156050 2026-02-24T18:35:37.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.549+0000 7f6aa27fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000c790 con 0x7f6ab4156050 2026-02-24T18:35:37.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 10 monclient: dump: 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6aa27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 5 monclient: authenticate success, global_id 4568 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 10 monclient: _renew_subs 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab40c8160 con 0x7f6ab4156050 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 10 client.4568.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 10 monclient: _renew_subs 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab40c8720 con 0x7f6ab4156050 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.550+0000 7f6abbf68640 1 librados: init done 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6abbf68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6abbf68640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6abbf68640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab40c7eb0 con 0x7f6ab4156050 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab0010070 con 0x7f6ab4156050 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000cb30 con 0x7f6ab4156050 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa27fc640 10 monclient: dump: 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.557 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.551+0000 7f6aa3fff640 4 set_mon_vals no callback set 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.554+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f63ac003e20 con 0x7f63b00c2fd0 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.554+0000 7f63a57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.554+0000 7f63a57fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.566+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8f2401d030 con 0x7f8f30155c70 2026-02-24T18:35:37.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.567+0000 7f8f1dffb640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 0x7f8f0007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.567+0000 7f8f356d1640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 0x7f8f0007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.567+0000 7f8f356d1640 10 monclient: get_auth_request con 0x7f8f00078e00 auth_method 0 2026-02-24T18:35:37.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.567+0000 7f8f1dffb640 10 client.4576.objecter ms_handle_connect 0x7f8f00078e00 2026-02-24T18:35:37.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.567+0000 7f8f356d1640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 0x7f8f0007b2c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006fd0 tx=0x7f8f20006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.568+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f8f2401dde0 con 0x7f8f30155c70 2026-02-24T18:35:37.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.568+0000 7f8f1dffb640 10 client.4576.objecter ms_dispatch2 0x7f8f30000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.568+0000 7f8f1dffb640 3 client.4576.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.568+0000 7f8f1dffb640 3 client.4576.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.568+0000 7f8f1dffb640 20 client.4576.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.575+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8f24024080 con 0x7f8f30155c70 2026-02-24T18:35:37.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.575+0000 7f8f1dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.575+0000 7f8f1dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.578+0000 7fe969ffb640 1 -- 10.20.193.18:0/85315979 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe978061be0 con 0x7fe97c15d730 2026-02-24T18:35:37.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.578+0000 7fe969ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-24T18:35:37.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.578+0000 7fe969ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.580 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:37.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 10 librados: watch_flush enter 2026-02-24T18:35:37.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 10 librados: watch_flush exit 2026-02-24T18:35:37.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 20 client.4564.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 10 client.4564.objecter successfully canceled tick 2026-02-24T18:35:37.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 msgr2=0x7fe94c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 0x7fe94c07b310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe97c08f510 tx=0x7fe96c031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 10 monclient: shutdown 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 msgr2=0x7fe97c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe97800b9d0 tx=0x7fe97800bea0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 shutdown_connections 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe94c078e50 0x7fe94c07b310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe97c15d730 0x7fe97c0c6960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe97c15d2c0 0x7fe97c0ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 --2- 10.20.193.18:0/85315979 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe97c1527f0 0x7fe97c1f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 >> 10.20.193.18:0/85315979 conn(0x7fe97c0ce730 msgr2=0x7fe97c0c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 shutdown_connections 2026-02-24T18:35:37.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 -- 10.20.193.18:0/85315979 wait complete. 2026-02-24T18:35:37.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.581+0000 7fe98329f640 1 librados: shutdown 2026-02-24T18:35:37.585 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.587+0000 7f660a258640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.587+0000 7f660a258640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.587+0000 7f660a258640 10 monclient: monmap: 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.588561+0000 2026-02-24T18:35:37.589 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.588561+0000 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.590 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 Processor -- start 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 -- start start 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient: init 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient: _add_conns ranks=[2,4,5,1,3,0] 2026-02-24T18:35:37.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f66040c90d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66040c94b0 0x7f6604158950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): picked mon.noname-b con 0x7f66040c94b0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6604158e90 0x7f6604074250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): picked mon.noname-c con 0x7f6604158e90 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6604073300 con 0x7f66040c90d0 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f660415ce70 con 0x7f66040c94b0 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6604073540 con 0x7f6604158e90 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.588+0000 7f660a258640 10 monclient(hunting): authenticate will time out at 1481.269932s 2026-02-24T18:35:37.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66040c94b0 0x7f6604158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f66040c94b0 0x7f6604158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53774/0 (socket says 10.20.193.18:53774) 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 1 -- 10.20.193.18:0/3759999797 learned_addr learned my addr 10.20.193.18:0/3759999797 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): get_auth_request con 0x7f66040c94b0 auth_method 0 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.4 2026-02-24T18:35:37.596 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.589+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 10 monclient(hunting): handle_auth_done global_id 4579 payload 274 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 1 -- 10.20.193.18:0/3759999797 >> v2:10.20.193.110:3300/0 conn(0x7f6604158e90 msgr2=0x7f6604074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 1 --2- 10.20.193.18:0/3759999797 >> v2:10.20.193.110:3300/0 conn(0x7f6604158e90 0x7f6604074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 1 -- 10.20.193.18:0/3759999797 >> v1:10.20.193.110:6789/0 conn(0x7f66040c90d0 legacy=0x7f6604072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.590+0000 7f6602ffd640 1 -- 10.20.193.18:0/3759999797 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6604159440 con 0x7f66040c94b0 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.591+0000 7f6602ffd640 1 --2- 10.20.193.18:0/3759999797 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604158950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f660406d740 tx=0x7f65f002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dbea18ba5c6bd3a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 1 -- 10.20.193.18:0/3759999797 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f003c070 con 0x7f66040c94b0 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: dump: 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.603 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 5 monclient: authenticate success, global_id 4579 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 1 -- 10.20.193.18:0/3759999797 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f002fb40 con 0x7f66040c94b0 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 1 -- 10.20.193.18:0/3759999797 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0037440 con 0x7f66040c94b0 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 4 set_mon_vals no callback set 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f66027fc640 10 monclient: dump: 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 10 monclient: shutdown 2026-02-24T18:35:37.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 -- 10.20.193.18:0/3759999797 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 msgr2=0x7f6604158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 --2- 10.20.193.18:0/3759999797 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604158950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f660406d740 tx=0x7f65f002eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 -- 10.20.193.18:0/3759999797 shutdown_connections 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 --2- 10.20.193.18:0/3759999797 >> v2:10.20.193.110:3300/0 conn(0x7f6604158e90 0x7f6604074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 --2- 10.20.193.18:0/3759999797 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604158950 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 -- 10.20.193.18:0/3759999797 >> 10.20.193.18:0/3759999797 conn(0x7f66040cd780 msgr2=0x7f66040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 -- 10.20.193.18:0/3759999797 shutdown_connections 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.593+0000 7f660a258640 1 -- 10.20.193.18:0/3759999797 wait complete. 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 10 monclient: monmap: 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.606 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 librados: starting msgr at 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 librados: starting objecter 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 Processor -- start 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 -- start start 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 librados: setting wanted keys 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 1 librados: calling monclient init 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.594+0000 7f660a258640 10 monclient: init 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66040c90d0 0x7f6604156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): picked mon.noname-c con 0x7f66040c90d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): picked mon.noname-b con 0x7f66040c94b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6604158e90 0x7f66040cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): picked mon.noname-a con 0x7f6604158e90 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6604073a70 con 0x7f6604158e90 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6604073790 con 0x7f66040c94b0 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6604073d50 con 0x7f66040c90d0 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f660a258640 10 monclient(hunting): authenticate will time out at 1481.276135s 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6603fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6604158e90 0x7f66040cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53778/0 (socket says 10.20.193.18:53778) 2026-02-24T18:35:37.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 1 -- 10.20.193.18:0/2556224315 learned_addr learned my addr 10.20.193.18:0/2556224315 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): get_auth_request con 0x7f66040c94b0 auth_method 0 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6603fff640 10 monclient(hunting): get_auth_request con 0x7f6604158e90 auth_method 0 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6603fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6603fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.595+0000 7f6602ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.596+0000 7f6603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.596+0000 7f6603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.596+0000 7f6603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 10 monclient(hunting): handle_auth_done global_id 4582 payload 824 2026-02-24T18:35:37.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 -- 10.20.193.18:0/2556224315 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66040c90d0 msgr2=0x7f6604156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66040c90d0 0x7f6604156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 -- 10.20.193.18:0/2556224315 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6604158e90 msgr2=0x7f66040cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6604158e90 0x7f66040cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.597+0000 7f6602ffd640 1 -- 10.20.193.18:0/2556224315 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6604143110 con 0x7f66040c94b0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f6602ffd640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009a70 tx=0x7f65f00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 client.?.objecter ms_handle_connect 0x7f66040c94b0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0044070 con 0x7f66040c94b0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 monclient: dump: 2026-02-24T18:35:37.610 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f65dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f660a258640 5 monclient: authenticate success, global_id 4582 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f660a258640 10 monclient: _renew_subs 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f660a258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.598+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6604153850 con 0x7f66040c94b0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 client.4582.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 monclient: _renew_subs 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6604153e10 con 0x7f66040c94b0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 1 librados: init done 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66041f6000 con 0x7f66040c94b0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f002fb40 con 0x7f66040c94b0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f004e060 con 0x7f66040c94b0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f6601ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:37.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.599+0000 7f65dffff640 10 monclient: dump: 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.613 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.613 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:35:37.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.641+0000 7f63b6d7e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:37.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.641+0000 7f63b6d7e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:37.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.641+0000 7f63b6d7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.641+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f63b0158a40 con 0x7f63b00c2fd0 2026-02-24T18:35:37.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.643+0000 7f63a57fa640 1 -- 10.20.193.18:0/3730499904 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f63ac086c80 con 0x7f63b00c2fd0 2026-02-24T18:35:37.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.643+0000 7f63a57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-24T18:35:37.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.643+0000 7f63a57fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.645 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:37.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 10 librados: watch_flush enter 2026-02-24T18:35:37.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 10 librados: watch_flush exit 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 20 client.4608.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 10 client.4608.objecter successfully canceled tick 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 msgr2=0x7f638007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 0x7f638007b310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63a0002820 tx=0x7f63a00027b0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 10 monclient: shutdown 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 msgr2=0x7f63b00cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c450 tx=0x7f63ac002e40 comp rx=0 tx=0).stop 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 shutdown_connections 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6380078e50 0x7f638007b310 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f63b00c3ad0 0x7f63b00c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f63b00c2fd0 0x7f63b00cc1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 --2- 10.20.193.18:0/3730499904 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f639c009ab0 0x7f63b0149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 >> 10.20.193.18:0/3730499904 conn(0x7f63b00cd780 msgr2=0x7f63b00cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 shutdown_connections 2026-02-24T18:35:37.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 -- 10.20.193.18:0/3730499904 wait complete. 2026-02-24T18:35:37.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.645+0000 7f63b6d7e640 1 librados: shutdown 2026-02-24T18:35:37.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.12 2026-02-24T18:35:37.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.655+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6ab001d030 con 0x7f6ab4156050 2026-02-24T18:35:37.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.655+0000 7f42d3bf7640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.655+0000 7f42d3bf7640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.655+0000 7f42d3bf7640 10 monclient: monmap: 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.656687+0000 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.656687+0000 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.658 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.659 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.659 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.659 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.659 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.655+0000 7f6aa27fc640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 0x7f6a8407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f6ab9cdd640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 0x7f6a8407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f6ab9cdd640 10 monclient: get_auth_request con 0x7f6a84078e00 auth_method 0 2026-02-24T18:35:37.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 1 Processor -- start 2026-02-24T18:35:37.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 1 -- start start 2026-02-24T18:35:37.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient: init 2026-02-24T18:35:37.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f6ab9cdd640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 0x7f6a8407b2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4072db0 tx=0x7f6aa4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f6aa27fc640 10 client.4568.objecter ms_handle_connect 0x7f6a84078e00 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient: _add_conns ranks=[2,5,0,1,3,4] 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f42cc1527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f42cc073530 0x7f42cc15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-c con 0x7f42cc073530 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f42cc15f900 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.656+0000 7f42d3bf7640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42cc0739a0 con 0x7f42cc15f900 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d3bf7640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42cc15fd90 con 0x7f42cc1527f0 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d3bf7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d3bf7640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f42cc074b10 con 0x7f42cc073530 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d3bf7640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d3bf7640 10 monclient(hunting): authenticate will time out at 1481.338010s 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f6ab001cdb0 con 0x7f6ab4156050 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f6aa27fc640 10 client.4568.objecter ms_dispatch2 0x7f6ab4000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f6aa27fc640 3 client.4568.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f6aa27fc640 3 client.4568.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f6aa27fc640 20 client.4568.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d216d640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f42cc15f900 0x7f42cc0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58762/0 (socket says 10.20.193.18:58762) 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.657+0000 7f42d216d640 1 -- 10.20.193.18:0/1674754444 learned_addr learned my addr 10.20.193.18:0/1674754444 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834333295 0 0) 0x7f42c8002ca0 con 0x7f42cc15f900 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 10 monclient(hunting): _negotiate 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 10 monclient(hunting): my global_id is 4614 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.658+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b4003620 con 0x7f42cc15f900 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 655335956 0 0) 0x7f42b4003620 con 0x7f42cc15f900 2026-02-24T18:35:37.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 >> v2:10.20.193.110:3300/0 conn(0x7f42cc073530 msgr2=0x7f42cc15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 --2- 10.20.193.18:0/1674754444 >> v2:10.20.193.110:3300/0 conn(0x7f42cc073530 0x7f42cc15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 >> v1:10.20.193.110:6789/0 conn(0x7f42cc1527f0 legacy=0x7f42cc15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc15d9d0 con 0x7f42cc15f900 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f42c8003450 con 0x7f42cc15f900 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:35:37.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: dump: 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 >> v1:10.20.193.18:6789/0 conn(0x7f42cc15f900 legacy=0x7f42cc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 0x7f42b4003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): picked mon.a con 0x7f42b4003ba0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42b4004900 0x7f42b4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): picked mon.c con 0x7f42b4004900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4005510 0x7f42b400bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): picked mon.b con 0x7f42b4005510 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc074b10 con 0x7f42b4003ba0 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42b400c450 con 0x7f42b4005510 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f42b400c650 con 0x7f42b4004900 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.660+0000 7f42d096a640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 0x7f42b4003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): get_auth_request con 0x7f42b4003ba0 auth_method 0 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.662+0000 7f42d196c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f8f3795c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f8f3795c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f8f3795c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f8f300cc590 con 0x7f8f30155c70 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6ab00620c0 con 0x7f6ab4156050 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f6aa27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f6aa27fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.663+0000 7f42d196c640 10 monclient(hunting): handle_auth_done global_id 4614 payload 293 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 -- 10.20.193.18:0/1674754444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42b4004900 msgr2=0x7f42b4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42b4004900 0x7f42b4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 -- 10.20.193.18:0/1674754444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4005510 msgr2=0x7f42b400bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4005510 0x7f42b400bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 monclient: found mon.a 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d216d640 1 -- 10.20.193.18:0/1674754444 reap_dead start 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 1 -- 10.20.193.18:0/1674754444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b4003620 con 0x7f42b4003ba0 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d196c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d3bf7640 5 monclient: authenticate success, global_id 4614 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.664+0000 7f42d3bf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.665+0000 7f42d196c640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 0x7f42b4003fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42bc002f60 tx=0x7f42bc02ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c9ba779ebdd099 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc002b30 con 0x7f42b4003ba0 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: dump: 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.678 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42bc033070 con 0x7f42b4003ba0 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d3bf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 1 -- 10.20.193.18:0/1674754444 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0054d0 con 0x7f42b4003ba0 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d096a640 10 monclient: dump: 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.679 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d3bf7640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.666+0000 7f42d3bf7640 4 set_mon_vals no callback set 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 10 monclient: shutdown 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 msgr2=0x7f42b4003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 0x7f42b4003fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42bc002f60 tx=0x7f42bc02ecb0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1674754444 shutdown_connections 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1674754444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42b4003ba0 0x7f42b4003fa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1674754444 >> 10.20.193.18:0/1674754444 conn(0x7f42cc0ce730 msgr2=0x7f42cc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1674754444 shutdown_connections 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1674754444 wait complete. 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.667+0000 7f42d3bf7640 10 monclient: monmap: 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.668+0000 7f42d3bf7640 1 librados: starting msgr at 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.668+0000 7f42d3bf7640 1 librados: starting objecter 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.668+0000 7f42d3bf7640 1 Processor -- start 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 -- start start 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 librados: setting wanted keys 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 librados: calling monclient init 2026-02-24T18:35:37.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient: init 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4003ba0 0x7f42cc0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-b con 0x7f42b4003ba0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 0x7f42cc1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-a con 0x7f42cc0ca390 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 0x7f42cc148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): picked mon.noname-c con 0x7f42cc1f60c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0741e0 con 0x7f42cc0ca390 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc073f00 con 0x7f42b4003ba0 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0744c0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d3bf7640 10 monclient(hunting): authenticate will time out at 1481.350627s 2026-02-24T18:35:37.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d116b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 0x7f42cc1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.669+0000 7f42d216d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 0x7f42cc148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 0x7f42cc1f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42090/0 (socket says 10.20.193.18:42090) 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 1 -- 10.20.193.18:0/1551668174 learned_addr learned my addr 10.20.193.18:0/1551668174 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): get_auth_request con 0x7f42cc0ca390 auth_method 0 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): get_auth_request con 0x7f42cc1f60c0 auth_method 0 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d116b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.670+0000 7f42d216d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 10 monclient(hunting): handle_auth_done global_id 4571 payload 824 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 -- 10.20.193.18:0/1551668174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4003ba0 msgr2=0x7f42cc0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4003ba0 0x7f42cc0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 -- 10.20.193.18:0/1551668174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 msgr2=0x7f42cc1f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 0x7f42cc1f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 monclient: found mon.noname-c 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.671+0000 7f42d216d640 1 -- 10.20.193.18:0/1551668174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc1f4ae0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.672+0000 7f42d216d640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 0x7f42cc148120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42cc08eb00 tx=0x7f42c800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.672+0000 7f42ba7fc640 10 client.?.objecter ms_handle_connect 0x7f42cc1f60c0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.672+0000 7f42ba7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8018070 con 0x7f42cc1f60c0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: dump: 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.684 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c800ad00 con 0x7f42cc1f60c0 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8007270 con 0x7f42cc1f60c0 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42bbfff640 4 set_mon_vals no callback set 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42ba7fc640 10 monclient: dump: 2026-02-24T18:35:37.685 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 5 monclient: authenticate success, global_id 4571 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 10 monclient: _renew_subs 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc1f4dc0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 10 client.4571.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 10 monclient: _renew_subs 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc1f5380 con 0x7f42cc1f60c0 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.674+0000 7f42d3bf7640 1 librados: init done 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.675+0000 7f42d3bf7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.675+0000 7f42d3bf7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.675+0000 7f42d3bf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.675+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc0747a0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.687 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 10 monclient: monmap: 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.682274+0000 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.682274+0000 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.689 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 1 Processor -- start 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 1 -- start start 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.681+0000 7f1c4a3fe640 10 monclient: init 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient: _add_conns ranks=[3,2,5,4,1,0] 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c44155ef0 0x7f1c4415c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-a con 0x7f1c44155ef0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c441562d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1c440cde80 0x7f1c440c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-c con 0x7f1c440cde80 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c440ce3b0 con 0x7f1c441562d0 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f1c4415cdb0 con 0x7f1c44155ef0 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1c440d2730 con 0x7f1c440cde80 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c4a3fe640 10 monclient(hunting): authenticate will time out at 1481.363420s 2026-02-24T18:35:37.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c44155ef0 0x7f1c4415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f1c44155ef0 0x7f1c4415c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42092/0 (socket says 10.20.193.18:42092) 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 1 -- 10.20.193.18:0/421292448 learned_addr learned my addr 10.20.193.18:0/421292448 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 10 monclient(hunting): get_auth_request con 0x7f1c44155ef0 auth_method 0 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.682+0000 7f1c43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.683+0000 7f1c43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.683+0000 7f1c43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.683+0000 7f1c43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 10 monclient(hunting): handle_auth_done global_id 4620 payload 274 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 1 -- 10.20.193.18:0/421292448 >> v2:10.20.193.110:3300/0 conn(0x7f1c440cde80 msgr2=0x7f1c440c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 1 --2- 10.20.193.18:0/421292448 >> v2:10.20.193.110:3300/0 conn(0x7f1c440cde80 0x7f1c440c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 1 -- 10.20.193.18:0/421292448 >> v1:10.20.193.110:6789/0 conn(0x7f1c441562d0 legacy=0x7f1c440cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 1 monclient: found mon.noname-a 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.684+0000 7f1c43fff640 1 -- 10.20.193.18:0/421292448 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c440ce690 con 0x7f1c44155ef0 2026-02-24T18:35:37.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.685+0000 7f1c43fff640 1 --2- 10.20.193.18:0/421292448 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c4415c870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c440cd930 tx=0x7f1c3002ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f10392047ec8669 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.685+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f42c8007410 con 0x7f42cc1f60c0 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.686+0000 7f42ba7fc640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 0x7f429c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.686+0000 7f42d196c640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 0x7f429c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.686+0000 7f42d196c640 10 monclient: get_auth_request con 0x7f429c078e00 auth_method 0 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.686+0000 7f42d196c640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 0x7f429c07b2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42bc037000 tx=0x7f42bc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.686+0000 7f42ba7fc640 10 client.4571.objecter ms_handle_connect 0x7f429c078e00 2026-02-24T18:35:37.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 1 -- 10.20.193.18:0/421292448 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3003c070 con 0x7f1c44155ef0 2026-02-24T18:35:37.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:37.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: dump: 2026-02-24T18:35:37.693 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 5 monclient: authenticate success, global_id 4620 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 1 -- 10.20.193.18:0/421292448 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3002fbd0 con 0x7f1c44155ef0 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 4 set_mon_vals no callback set 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 1 -- 10.20.193.18:0/421292448 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30037440 con 0x7f1c44155ef0 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c42ffd640 10 monclient: dump: 2026-02-24T18:35:37.694 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 10 monclient: shutdown 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/421292448 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 msgr2=0x7f1c4415c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/421292448 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c4415c870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c440cd930 tx=0x7f1c3002ef60 comp rx=0 tx=0).stop 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/421292448 shutdown_connections 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/421292448 >> v2:10.20.193.110:3300/0 conn(0x7f1c440cde80 0x7f1c440c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/421292448 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c4415c870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/421292448 >> 10.20.193.18:0/421292448 conn(0x7f1c4406d8c0 msgr2=0x7f1c4406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/421292448 shutdown_connections 2026-02-24T18:35:37.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/421292448 wait complete. 2026-02-24T18:35:37.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f42c805d980 con 0x7f42cc1f60c0 2026-02-24T18:35:37.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f42ba7fc640 10 client.4571.objecter ms_dispatch2 0x7f42cc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f42ba7fc640 3 client.4571.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.687+0000 7f42ba7fc640 3 client.4571.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f42ba7fc640 20 client.4571.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 10 monclient: monmap: 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.697 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 librados: starting msgr at 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 librados: starting objecter 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 Processor -- start 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 -- start start 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 librados: setting wanted keys 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 1 librados: calling monclient init 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.688+0000 7f1c4a3fe640 10 monclient: init 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c440cde80 0x7f1c4415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-b con 0x7f1c440cde80 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-a con 0x7f1c44155ef0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c441562d0 0x7f1c44073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): picked mon.noname-c con 0x7f1c441562d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440cef30 con 0x7f1c44155ef0 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440cec50 con 0x7f1c440cde80 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440c95c0 con 0x7f1c441562d0 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c4a3fe640 10 monclient(hunting): authenticate will time out at 1481.370137s 2026-02-24T18:35:37.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42094/0 (socket says 10.20.193.18:42094) 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 1 -- 10.20.193.18:0/185768286 learned_addr learned my addr 10.20.193.18:0/185768286 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): get_auth_request con 0x7f1c44155ef0 auth_method 0 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.689+0000 7f1c437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 10 monclient(hunting): handle_auth_done global_id 4623 payload 824 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 -- 10.20.193.18:0/185768286 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c441562d0 msgr2=0x7f1c44073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c441562d0 0x7f1c44073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 -- 10.20.193.18:0/185768286 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c440cde80 msgr2=0x7f1c4415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c440cde80 0x7f1c4415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.691+0000 7f1c437fe640 1 -- 10.20.193.18:0/185768286 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c441f72b0 con 0x7f1c44155ef0 2026-02-24T18:35:37.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.692+0000 7f1c437fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400b4d0 tx=0x7f1c3400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.692+0000 7f1c40ff9640 10 client.?.objecter ms_handle_connect 0x7f1c44155ef0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.692+0000 7f1c40ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34004480 con 0x7f1c44155ef0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: dump: 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34015070 con 0x7f1c44155ef0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34002920 con 0x7f1c44155ef0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 5 monclient: authenticate success, global_id 4623 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _renew_subs 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c427fc640 4 set_mon_vals no callback set 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c441f7590 con 0x7f1c44155ef0 2026-02-24T18:35:37.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c40ff9640 10 monclient: dump: 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 client.4623.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _renew_subs 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c441f7b50 con 0x7f1c44155ef0 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 1 librados: init done 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.694+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c440c99d0 con 0x7f1c44155ef0 2026-02-24T18:35:37.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.704+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f65f003f050 con 0x7f66040c94b0 2026-02-24T18:35:37.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.705+0000 7f65dffff640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 0x7f65d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.705+0000 7f66037fe640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 0x7f65d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.705+0000 7f66037fe640 10 monclient: get_auth_request con 0x7f65d4078e00 auth_method 0 2026-02-24T18:35:37.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.705+0000 7f66037fe640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 0x7f65d407b2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f660414fef0 tx=0x7f65f8009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.705+0000 7f65dffff640 10 client.4582.objecter ms_handle_connect 0x7f65d4078e00 2026-02-24T18:35:37.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.706+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1c34002ac0 con 0x7f1c44155ef0 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.706+0000 7f1c40ff9640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 0x7f1c1407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f65f0082f30 con 0x7f66040c94b0 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f65dffff640 10 client.4582.objecter ms_dispatch2 0x7f6604000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f65dffff640 3 client.4582.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f65dffff640 3 client.4582.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f1c43fff640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 0x7f1c1407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f1c43fff640 10 monclient: get_auth_request con 0x7f1c14078e00 auth_method 0 2026-02-24T18:35:37.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f65dffff640 20 client.4582.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f1c40ff9640 10 client.4623.objecter ms_handle_connect 0x7f1c14078e00 2026-02-24T18:35:37.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.707+0000 7f1c43fff640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 0x7f1c1407b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c4408e920 tx=0x7f1c30031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.709+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f1c3405e2e0 con 0x7f1c44155ef0 2026-02-24T18:35:37.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.709+0000 7f1c40ff9640 10 client.4623.objecter ms_dispatch2 0x7f1c44000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.709+0000 7f1c40ff9640 3 client.4623.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.709+0000 7f1c40ff9640 3 client.4623.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.709+0000 7f1c40ff9640 20 client.4623.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.713+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f66041f6000 con 0x7f66040c94b0 2026-02-24T18:35:37.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.713+0000 7f65dffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.713+0000 7f65dffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.727+0000 7f8f1dffb640 1 -- 10.20.193.18:0/2069898843 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f8f301ea1c0 con 0x7f8f30155c70 2026-02-24T18:35:37.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.727+0000 7f8f1dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-24T18:35:37.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.727+0000 7f8f1dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.729 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 10 librados: watch_flush enter 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 10 librados: watch_flush exit 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 20 client.4576.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 10 client.4576.objecter successfully canceled tick 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 msgr2=0x7f8f0007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 0x7f8f0007b2c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006fd0 tx=0x7f8f20006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 10 monclient: shutdown 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 msgr2=0x7f8f3015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b7b0 tx=0x7f8f2400bc80 comp rx=0 tx=0).stop 2026-02-24T18:35:37.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 shutdown_connections 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8f00078e00 0x7f8f0007b2c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8f30155c70 0x7f8f3015f340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8f3015d9d0 0x7f8f3014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 --2- 10.20.193.18:0/2069898843 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8f300ccb30 0x7f8f3015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 >> 10.20.193.18:0/2069898843 conn(0x7f8f300cc010 msgr2=0x7f8f3015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 shutdown_connections 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 -- 10.20.193.18:0/2069898843 wait complete. 2026-02-24T18:35:37.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.729+0000 7f8f3795c640 1 librados: shutdown 2026-02-24T18:35:37.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.15 2026-02-24T18:35:37.742 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.753+0000 7f6abbf68640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:37.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.753+0000 7f6abbf68640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:37.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.753+0000 7f6abbf68640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.753+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6ab4070210 con 0x7f6ab4156050 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.760+0000 7f0cd458f640 10 monclient: get_monmap_and_config 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.760+0000 7f0cd458f640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.760+0000 7f0cd458f640 10 monclient: monmap: 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:37.761726+0000 2026-02-24T18:35:37.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:37.761726+0000 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:37.763 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.761+0000 7f0cd458f640 1 Processor -- start 2026-02-24T18:35:37.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.761+0000 7f0cd458f640 1 -- start start 2026-02-24T18:35:37.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.761+0000 7f0cd458f640 10 monclient: init 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient: _add_conns ranks=[0,4,2,1,5,3] 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0ccc15b860 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0ccc0740a0 0x7f0ccc15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-b con 0x7f0ccc0740a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0ccc072480 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ccc072d20 con 0x7f0ccc15b860 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ccc073000 con 0x7f0ccc072480 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0ccc0732e0 con 0x7f0ccc0740a0 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd458f640 10 monclient(hunting): authenticate will time out at 1481.443105s 2026-02-24T18:35:37.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0ccc0740a0 0x7f0ccc15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0ccc0740a0 0x7f0ccc15bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53818/0 (socket says 10.20.193.18:53818) 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 1 -- 10.20.193.18:0/406041209 learned_addr learned my addr 10.20.193.18:0/406041209 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 10 monclient(hunting): get_auth_request con 0x7f0ccc0740a0 auth_method 0 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.762+0000 7f0cd1b03640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient(hunting): handle_auth_done global_id 4585 payload 274 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 1 -- 10.20.193.18:0/406041209 >> v1:10.20.193.110:6789/0 conn(0x7f0ccc072480 legacy=0x7f0ccc0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 1 -- 10.20.193.18:0/406041209 >> v1:10.20.193.18:6789/0 conn(0x7f0ccc15b860 legacy=0x7f0ccc1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:37.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.763+0000 7f0cd1b03640 1 -- 10.20.193.18:0/406041209 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc0735c0 con 0x7f0ccc0740a0 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.764+0000 7f0cd1b03640 1 --2- 10.20.193.18:0/406041209 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc15bc40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00099a0 tx=0x7f0cc002ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4ae8d020cccf2e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 1 -- 10.20.193.18:0/406041209 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc003c070 con 0x7f0ccc0740a0 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: dump: 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.771 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 1 -- 10.20.193.18:0/406041209 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc002fb80 con 0x7f0ccc0740a0 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 5 monclient: authenticate success, global_id 4585 2026-02-24T18:35:37.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 1 -- 10.20.193.18:0/406041209 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0037440 con 0x7f0ccc0740a0 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 4 set_mon_vals no callback set 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd1302640 10 monclient: dump: 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.773 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 10 monclient: shutdown 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 -- 10.20.193.18:0/406041209 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc0740a0 msgr2=0x7f0ccc15bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 --2- 10.20.193.18:0/406041209 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc15bc40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00099a0 tx=0x7f0cc002ef50 comp rx=0 tx=0).stop 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 -- 10.20.193.18:0/406041209 shutdown_connections 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 --2- 10.20.193.18:0/406041209 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc15bc40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 -- 10.20.193.18:0/406041209 >> 10.20.193.18:0/406041209 conn(0x7f0ccc0cd780 msgr2=0x7f0ccc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 -- 10.20.193.18:0/406041209 shutdown_connections 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.766+0000 7f0cd458f640 1 -- 10.20.193.18:0/406041209 wait complete. 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient: build_initial_monmap 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient: monmap: 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:37.774 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 librados: starting msgr at 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 librados: starting objecter 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 Processor -- start 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 -- start start 2026-02-24T18:35:37.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 librados: setting wanted keys 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 librados: calling monclient init 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient: init 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-b con 0x7f0ccc072480 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc0c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-c con 0x7f0ccc0740a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0ccc15b860 0x7f0ccc0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): picked mon.noname-a con 0x7f0ccc15b860 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc0728f0 con 0x7f0ccc15b860 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc073b80 con 0x7f0ccc072480 2026-02-24T18:35:37.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc072a70 con 0x7f0ccc0740a0 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.767+0000 7f0cd458f640 10 monclient(hunting): authenticate will time out at 1481.448869s 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53834/0 (socket says 10.20.193.18:53834) 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 1 -- 10.20.193.18:0/3759079773 learned_addr learned my addr 10.20.193.18:0/3759079773 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): get_auth_request con 0x7f0ccc072480 auth_method 0 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:37.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.768+0000 7f0cd2304640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 10 monclient(hunting): handle_auth_done global_id 4588 payload 824 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 -- 10.20.193.18:0/3759079773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0ccc0740a0 msgr2=0x7f0ccc0c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc0c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 -- 10.20.193.18:0/3759079773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0ccc15b860 msgr2=0x7f0ccc0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0ccc15b860 0x7f0ccc0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 10 monclient: _finish_hunting 0 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 monclient: found mon.noname-b 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.769+0000 7f0cd2304640 1 -- 10.20.193.18:0/3759079773 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc0c2660 con 0x7f0ccc072480 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.770+0000 7f0cd2304640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00b4d0 tx=0x7f0cbc00b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.770+0000 7f0cb2ffd640 10 client.?.objecter ms_handle_connect 0x7f0ccc072480 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.770+0000 7f0cb2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc004480 con 0x7f0ccc072480 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: dump: 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.780 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc015070 con 0x7f0ccc072480 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc002920 con 0x7f0ccc072480 2026-02-24T18:35:37.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cb2ffd640 10 monclient: dump: 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd0b01640 4 set_mon_vals no callback set 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 5 monclient: authenticate success, global_id 4588 2026-02-24T18:35:37.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _renew_subs 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc0c2940 con 0x7f0ccc072480 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 client.4588.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _renew_subs 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc0c2f00 con 0x7f0ccc072480 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 1 librados: init done 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.772+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc15bcc0 con 0x7f0ccc072480 2026-02-24T18:35:37.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.784+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0cbc002ac0 con 0x7f0ccc072480 2026-02-24T18:35:37.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.784+0000 7f0cb2ffd640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 0x7f0c9c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:37.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.784+0000 7f0cd1b03640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 0x7f0c9c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:37.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.784+0000 7f0cd1b03640 10 monclient: get_auth_request con 0x7f0c9c078e00 auth_method 0 2026-02-24T18:35:37.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cd1b03640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 0x7f0c9c07b2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc075110 tx=0x7f0cc0031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:37.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 10 client.4588.objecter ms_handle_connect 0x7f0c9c078e00 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f0cbc05e2c0 con 0x7f0ccc072480 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 10 client.4588.objecter ms_dispatch2 0x7f0ccc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 3 client.4588.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 3 client.4588.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f42cc0739a0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f42ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f42ba7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.785+0000 7f0cb2ffd640 20 client.4588.objecter dump_active .. 0 homeless 2026-02-24T18:35:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.797+0000 7f660a258640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.797+0000 7f660a258640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.797+0000 7f660a258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.797+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f66040c8530 con 0x7f66040c94b0 2026-02-24T18:35:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.800+0000 7f6aa27fc640 1 -- 10.20.193.18:0/1201956293 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ab00619c0 con 0x7f6ab4156050 2026-02-24T18:35:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.800+0000 7f6aa27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-24T18:35:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.800+0000 7f6aa27fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.802 INFO:teuthology.orchestra.run.trial018.stdout:94489280516 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 10 librados: watch_flush enter 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 10 librados: watch_flush exit 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 20 client.4568.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 10 client.4568.objecter successfully canceled tick 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 msgr2=0x7f6a8407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 0x7f6a8407b2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4072db0 tx=0x7f6aa4031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 10 monclient: shutdown 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 msgr2=0x7f6ab4150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.802+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b9c0 tx=0x7f6ab000be90 comp rx=0 tx=0).stop 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 shutdown_connections 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a84078e00 0x7f6a8407b2c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6ab4155c70 0x7f6ab415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6ab4156050 0x7f6ab4150f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 --2- 10.20.193.18:0/1201956293 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6ab40ccb30 0x7f6ab41e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 >> 10.20.193.18:0/1201956293 conn(0x7f6ab40cc010 msgr2=0x7f6ab406e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 shutdown_connections 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 -- 10.20.193.18:0/1201956293 wait complete. 2026-02-24T18:35:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.803+0000 7f6abbf68640 1 librados: shutdown 2026-02-24T18:35:37.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.810+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c340629a0 con 0x7f1c44155ef0 2026-02-24T18:35:37.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.810+0000 7f1c40ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.810+0000 7f1c40ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.1 2026-02-24T18:35:37.815 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.873+0000 7f42d3bf7640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:37.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.873+0000 7f42d3bf7640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:37.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.873+0000 7f42d3bf7640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:37.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.873+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f42cc1503f0 con 0x7f42cc1f60c0 2026-02-24T18:35:37.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.875+0000 7f42ba7fc640 1 -- 10.20.193.18:0/1551668174 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42c8061940 con 0x7f42cc1f60c0 2026-02-24T18:35:37.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.875+0000 7f42ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-24T18:35:37.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.875+0000 7f42ba7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.877 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 10 librados: watch_flush enter 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 10 librados: watch_flush exit 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 20 client.4571.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 10 client.4571.objecter successfully canceled tick 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 msgr2=0x7f429c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 0x7f429c07b2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42bc037000 tx=0x7f42bc031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 10 monclient: shutdown 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 msgr2=0x7f42cc148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 0x7f42cc148120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42cc08eb00 tx=0x7f42c800a430 comp rx=0 tx=0).stop 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 shutdown_connections 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f429c078e00 0x7f429c07b2c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f42cc1f60c0 0x7f42cc148120 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f42cc0ca390 0x7f42cc1f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 --2- 10.20.193.18:0/1551668174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f42b4003ba0 0x7f42cc0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 >> 10.20.193.18:0/1551668174 conn(0x7f42cc0ce730 msgr2=0x7f42cc0ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 shutdown_connections 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 -- 10.20.193.18:0/1551668174 wait complete. 2026-02-24T18:35:37.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.877+0000 7f42d3bf7640 1 librados: shutdown 2026-02-24T18:35:37.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.882+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0cbc062980 con 0x7f0ccc072480 2026-02-24T18:35:37.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.882+0000 7f0cb2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:37.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.882+0000 7f0cb2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:37.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.886+0000 7f65dffff640 1 -- 10.20.193.18:0/2556224315 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65f0086ef0 con 0x7f66040c94b0 2026-02-24T18:35:37.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.886+0000 7f65dffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-24T18:35:37.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.886+0000 7f65dffff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.888 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:37.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.8 2026-02-24T18:35:37.889 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 10 librados: watch_flush enter 2026-02-24T18:35:37.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 10 librados: watch_flush exit 2026-02-24T18:35:37.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 20 client.4582.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 10 client.4582.objecter successfully canceled tick 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 msgr2=0x7f65d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 0x7f65d407b2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f660414fef0 tx=0x7f65f8009250 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 10 monclient: shutdown 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 msgr2=0x7f6604142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009a70 tx=0x7f65f00047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 shutdown_connections 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f65d4078e00 0x7f65d407b2c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f66040c94b0 0x7f6604142bd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6604158e90 0x7f66040cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 --2- 10.20.193.18:0/2556224315 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f66040c90d0 0x7f6604156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 >> 10.20.193.18:0/2556224315 conn(0x7f66040cd780 msgr2=0x7f66040cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 shutdown_connections 2026-02-24T18:35:37.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 -- 10.20.193.18:0/2556224315 wait complete. 2026-02-24T18:35:37.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.888+0000 7f660a258640 1 librados: shutdown 2026-02-24T18:35:37.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.897+0000 7f1c4a3fe640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:37.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.897+0000 7f1c4a3fe640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:37.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.897+0000 7f1c4a3fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:37.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.897+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1c440cc370 con 0x7f1c44155ef0 2026-02-24T18:35:37.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.9 2026-02-24T18:35:37.900 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.902+0000 7f1c40ff9640 1 -- 10.20.193.18:0/185768286 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1c340622a0 con 0x7f1c44155ef0 2026-02-24T18:35:37.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.902+0000 7f1c40ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-24T18:35:37.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.902+0000 7f1c40ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.904 INFO:teuthology.orchestra.run.trial018.stdout:90194313221 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 10 librados: watch_flush enter 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 10 librados: watch_flush exit 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 20 client.4623.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 10 client.4623.objecter successfully canceled tick 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 msgr2=0x7f1c1407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 0x7f1c1407b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c4408e920 tx=0x7f1c30031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 10 monclient: shutdown 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 msgr2=0x7f1c441566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400b4d0 tx=0x7f1c3400b9a0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 shutdown_connections 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1c14078e00 0x7f1c1407b2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c441562d0 0x7f1c44073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c44155ef0 0x7f1c441566e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 --2- 10.20.193.18:0/185768286 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c440cde80 0x7f1c4415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 >> 10.20.193.18:0/185768286 conn(0x7f1c4406d8c0 msgr2=0x7f1c44153710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 shutdown_connections 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 -- 10.20.193.18:0/185768286 wait complete. 2026-02-24T18:35:37.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.904+0000 7f1c4a3fe640 1 librados: shutdown 2026-02-24T18:35:37.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313221 for osd.2 2026-02-24T18:35:37.915 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:37.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.967+0000 7f0cd458f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:37.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.967+0000 7f0cd458f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:37.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.967+0000 7f0cd458f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:37.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.967+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f0ccc15c5a0 con 0x7f0ccc072480 2026-02-24T18:35:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.969+0000 7f0cb2ffd640 1 -- 10.20.193.18:0/3759079773 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f0cbc062280 con 0x7f0ccc072480 2026-02-24T18:35:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.969+0000 7f0cb2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-24T18:35:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.969+0000 7f0cb2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:37.971 INFO:teuthology.orchestra.run.trial018.stdout:90194313221 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 10 librados: watch_flush enter 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 10 librados: watch_flush exit 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 20 client.4588.objecter shutdown clearing up homeless session... 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 10 client.4588.objecter successfully canceled tick 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 msgr2=0x7f0c9c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 0x7f0c9c07b2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc075110 tx=0x7f0cc0031000 comp rx=0 tx=0).stop 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 10 monclient: shutdown 2026-02-24T18:35:37.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 msgr2=0x7f0ccc14fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00b4d0 tx=0x7f0cbc00b9a0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 shutdown_connections 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0c9c078e00 0x7f0c9c07b2c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0ccc15b860 0x7f0ccc0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0ccc0740a0 0x7f0ccc0c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 --2- 10.20.193.18:0/3759079773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0ccc072480 0x7f0ccc14fe50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 >> 10.20.193.18:0/3759079773 conn(0x7f0ccc0cd780 msgr2=0x7f0ccc0c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 shutdown_connections 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 -- 10.20.193.18:0/3759079773 wait complete. 2026-02-24T18:35:37.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:37.970+0000 7f0cd458f640 1 librados: shutdown 2026-02-24T18:35:37.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313221 for osd.10 2026-02-24T18:35:37.981 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.061 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:35:38.071 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-24T18:35:38.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.131+0000 7f27b0ea3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.131+0000 7f27b0ea3640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.131+0000 7f27b0ea3640 10 monclient: monmap: 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.132331+0000 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.132331+0000 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.133 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.134 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.134 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.134 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.134 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.134 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.131+0000 7f27b0ea3640 1 Processor -- start 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 -- start start 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient: init 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient: _add_conns ranks=[5,0,4,3,1,2] 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f27ac152510 0x7f27ac15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-c con 0x7f27ac152510 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f27ac159b40 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f27ac15f8d0 0x7f27ac15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-b con 0x7f27ac15f8d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ac15d820 con 0x7f27ac159b40 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac15d600 con 0x7f27ac15f8d0 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac159f90 con 0x7f27ac152510 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.132+0000 7f27b0ea3640 10 monclient(hunting): authenticate will time out at 1481.813719s 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27aaffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f27ac159b40 0x7f27ac15f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58784/0 (socket says 10.20.193.18:58784) 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27aaffd640 1 -- 10.20.193.18:0/1705355535 learned_addr learned my addr 10.20.193.18:0/1705355535 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:38.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 1 --2- 10.20.193.18:0/1705355535 >> v2:10.20.193.110:3300/0 conn(0x7f27ac152510 0x7f27ac15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 1 --2- 10.20.193.18:0/1705355535 >> v2:10.20.193.32:3300/0 conn(0x7f27ac15f8d0 0x7f27ac15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): get_auth_request con 0x7f27ac152510 auth_method 0 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): get_auth_request con 0x7f27ac15f8d0 auth_method 0 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.139 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.133+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 1 -- 10.20.193.18:0/1705355535 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210068538 0 0) 0x7f27ac15d820 con 0x7f27ac159b40 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 10 monclient(hunting): my global_id is 4626 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27aa7fc640 1 -- 10.20.193.18:0/1705355535 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2790003620 con 0x7f27ac159b40 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_done global_id 4574 payload 274 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 1 -- 10.20.193.18:0/1705355535 >> v2:10.20.193.32:3300/0 conn(0x7f27ac15f8d0 msgr2=0x7f27ac15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 1 --2- 10.20.193.18:0/1705355535 >> v2:10.20.193.32:3300/0 conn(0x7f27ac15f8d0 0x7f27ac15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 1 -- 10.20.193.18:0/1705355535 >> v1:10.20.193.18:6789/0 conn(0x7f27ac159b40 legacy=0x7f27ac15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:38.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.134+0000 7f27ab7fe640 1 -- 10.20.193.18:0/1705355535 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac15db00 con 0x7f27ac152510 2026-02-24T18:35:38.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.135+0000 7f27ab7fe640 1 --2- 10.20.193.18:0/1705355535 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac15eba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0699f0 tx=0x7f279802ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8092db9c526f21 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 1 -- 10.20.193.18:0/1705355535 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279803c070 con 0x7f27ac152510 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 10 monclient: dump: 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.146 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27b0ea3640 5 monclient: authenticate success, global_id 4574 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.136+0000 7f27b0ea3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 1 -- 10.20.193.18:0/1705355535 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279802faf0 con 0x7f27ac152510 2026-02-24T18:35:38.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 1 -- 10.20.193.18:0/1705355535 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798037440 con 0x7f27ac152510 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 4 set_mon_vals no callback set 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27aa7fc640 10 monclient: dump: 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.148 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 10 monclient: shutdown 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1705355535 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 msgr2=0x7f27ac15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1705355535 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac15eba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0699f0 tx=0x7f279802ede0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1705355535 shutdown_connections 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1705355535 >> v2:10.20.193.32:3300/0 conn(0x7f27ac15f8d0 0x7f27ac15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1705355535 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac15eba0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1705355535 >> 10.20.193.18:0/1705355535 conn(0x7f27ac0ce330 msgr2=0x7f27ac0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1705355535 shutdown_connections 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.137+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1705355535 wait complete. 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient: monmap: 2026-02-24T18:35:38.149 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 librados: starting msgr at 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 librados: starting objecter 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 Processor -- start 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 -- start start 2026-02-24T18:35:38.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 librados: setting wanted keys 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 librados: calling monclient init 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient: init 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac1f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-c con 0x7f27ac152510 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27ac159b40 0x7f27ac14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-b con 0x7f27ac159b40 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): picked mon.noname-a con 0x7f27ac15f8d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac15e3a0 con 0x7f27ac15f8d0 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac15e0c0 con 0x7f27ac159b40 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac15e680 con 0x7f27ac152510 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27b0ea3640 10 monclient(hunting): authenticate will time out at 1481.819766s 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.138+0000 7f27abfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac1f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42116/0 (socket says 10.20.193.18:42116) 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 1 -- 10.20.193.18:0/1937639606 learned_addr learned my addr 10.20.193.18:0/1937639606 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): get_auth_request con 0x7f27ac15f8d0 auth_method 0 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): get_auth_request con 0x7f27ac152510 auth_method 0 2026-02-24T18:35:38.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27ac159b40 0x7f27ac14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): get_auth_request con 0x7f27ac159b40 auth_method 0 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.139+0000 7f27aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 10 monclient(hunting): handle_auth_done global_id 4629 payload 824 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 -- 10.20.193.18:0/1937639606 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 msgr2=0x7f27ac1f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac1f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 -- 10.20.193.18:0/1937639606 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27ac159b40 msgr2=0x7f27ac14e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27ac159b40 0x7f27ac14e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 monclient: found mon.noname-a 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.140+0000 7f27abfff640 1 -- 10.20.193.18:0/1937639606 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac1f3290 con 0x7f27ac15f8d0 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.141+0000 7f27abfff640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f27a000efc0 tx=0x7f27a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.141+0000 7f2787fff640 10 client.?.objecter ms_handle_connect 0x7f27ac15f8d0 2026-02-24T18:35:38.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.141+0000 7f2787fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0018020 con 0x7f27ac15f8d0 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: dump: 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.155 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a0014480 con 0x7f27ac15f8d0 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0014870 con 0x7f27ac15f8d0 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27a9ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f2787fff640 10 monclient: dump: 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.156 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 5 monclient: authenticate success, global_id 4629 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 10 monclient: _renew_subs 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac1f3570 con 0x7f27ac15f8d0 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 10 client.4629.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 10 monclient: _renew_subs 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac1f3b30 con 0x7f27ac15f8d0 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.143+0000 7f27b0ea3640 1 librados: init done 2026-02-24T18:35:38.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.144+0000 7f27b0ea3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.144+0000 7f27b0ea3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.144+0000 7f27b0ea3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.144+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ac1528f0 con 0x7f27ac15f8d0 2026-02-24T18:35:38.158 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.171+0000 7f53d3474640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.171+0000 7f53d3474640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.171+0000 7f53d3474640 10 monclient: monmap: 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.172626+0000 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.172626+0000 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.174 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.172+0000 7f53d3474640 1 Processor -- start 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.172+0000 7f53d3474640 1 -- start start 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.172+0000 7f53d3474640 10 monclient: init 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.172+0000 7f53d3474640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient: _add_conns ranks=[5,4,0,1,3,2] 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f53cc15b860 0x7f53cc15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-c con 0x7f53cc15b860 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f53cc0740a0 0x7f53cc1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-b con 0x7f53cc0740a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f53cc072480 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53cc072d20 con 0x7f53cc072480 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f53cc073000 con 0x7f53cc0740a0 2026-02-24T18:35:38.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f53cc0732e0 con 0x7f53cc15b860 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d3474640 10 monclient(hunting): authenticate will time out at 1481.854061s 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f53cc0740a0 0x7f53cc1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d11e9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f53cc15b860 0x7f53cc15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f53cc0740a0 0x7f53cc1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53872/0 (socket says 10.20.193.18:53872) 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 1 -- 10.20.193.18:0/1656237822 learned_addr learned my addr 10.20.193.18:0/1656237822 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 10 monclient(hunting): get_auth_request con 0x7f53cc0740a0 auth_method 0 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d09e8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d11e9640 10 monclient(hunting): get_auth_request con 0x7f53cc15b860 auth_method 0 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d11e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d11e9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.173+0000 7f53d11e9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d11e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d11e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d11e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient(hunting): handle_auth_done global_id 4597 payload 274 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 1 -- 10.20.193.18:0/1656237822 >> v2:10.20.193.110:3300/0 conn(0x7f53cc15b860 msgr2=0x7f53cc15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 1 --2- 10.20.193.18:0/1656237822 >> v2:10.20.193.110:3300/0 conn(0x7f53cc15b860 0x7f53cc15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 1 -- 10.20.193.18:0/1656237822 >> v1:10.20.193.18:6789/0 conn(0x7f53cc072480 legacy=0x7f53cc0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 1 monclient: found mon.noname-b 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.174+0000 7f53d09e8640 1 -- 10.20.193.18:0/1656237822 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc0735c0 con 0x7f53cc0740a0 2026-02-24T18:35:38.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.175+0000 7f53d09e8640 1 --2- 10.20.193.18:0/1656237822 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc0740a0 0x7f53cc1594f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009970 tx=0x7f53c002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc703574640964b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 1 -- 10.20.193.18:0/1656237822 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c003c070 con 0x7f53cc0740a0 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: dump: 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.184 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 1 -- 10.20.193.18:0/1656237822 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c002fb20 con 0x7f53cc0740a0 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 5 monclient: authenticate success, global_id 4597 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 1 -- 10.20.193.18:0/1656237822 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0037440 con 0x7f53cc0740a0 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53bbfff640 10 monclient: dump: 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.185 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 4 set_mon_vals no callback set 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 10 monclient: shutdown 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 -- 10.20.193.18:0/1656237822 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc0740a0 msgr2=0x7f53cc1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 --2- 10.20.193.18:0/1656237822 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc0740a0 0x7f53cc1594f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009970 tx=0x7f53c002eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 -- 10.20.193.18:0/1656237822 shutdown_connections 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 --2- 10.20.193.18:0/1656237822 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc0740a0 0x7f53cc1594f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 --2- 10.20.193.18:0/1656237822 >> v2:10.20.193.110:3300/0 conn(0x7f53cc15b860 0x7f53cc15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 -- 10.20.193.18:0/1656237822 >> 10.20.193.18:0/1656237822 conn(0x7f53cc0cd780 msgr2=0x7f53cc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 -- 10.20.193.18:0/1656237822 shutdown_connections 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.177+0000 7f53d3474640 1 -- 10.20.193.18:0/1656237822 wait complete. 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 10 monclient: monmap: 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.187 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 librados: starting msgr at 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 librados: starting objecter 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 Processor -- start 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 -- start start 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 librados: setting wanted keys 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 1 librados: calling monclient init 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.178+0000 7f53d3474640 10 monclient: init 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f53cc072480 0x7f53cc151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-a con 0x7f53cc072480 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53cc0740a0 0x7f53cc1e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-c con 0x7f53cc0740a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): picked mon.noname-b con 0x7f53cc15b860 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc159a30 con 0x7f53cc072480 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc073b80 con 0x7f53cc15b860 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc0728f0 con 0x7f53cc0740a0 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d3474640 10 monclient(hunting): authenticate will time out at 1481.860124s 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d09e8640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53cc0740a0 0x7f53cc1e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53874/0 (socket says 10.20.193.18:53874) 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 1 -- 10.20.193.18:0/1292392506 learned_addr learned my addr 10.20.193.18:0/1292392506 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 10 monclient(hunting): get_auth_request con 0x7f53cc15b860 auth_method 0 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d19ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d09e8640 10 monclient(hunting): get_auth_request con 0x7f53cc0740a0 auth_method 0 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d09e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d09e8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.179+0000 7f53d09e8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d19ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d19ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d19ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.180+0000 7f53d09e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 10 monclient(hunting): handle_auth_done global_id 4600 payload 824 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 -- 10.20.193.18:0/1292392506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53cc0740a0 msgr2=0x7f53cc1e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53cc0740a0 0x7f53cc1e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 -- 10.20.193.18:0/1292392506 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f53cc072480 msgr2=0x7f53cc151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f53cc072480 0x7f53cc151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 monclient: found mon.noname-b 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.181+0000 7f53d19ea640 1 -- 10.20.193.18:0/1292392506 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc158e90 con 0x7f53cc15b860 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.182+0000 7f53d19ea640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b9c0 tx=0x7f53c800be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.182+0000 7f53b9ffb640 10 client.?.objecter ms_handle_connect 0x7f53cc15b860 2026-02-24T18:35:38.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.182+0000 7f53b9ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.192 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8006e60 con 0x7f53cc15b860 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: dump: 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.194 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 5 monclient: authenticate success, global_id 4600 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 10 monclient: _renew_subs 2026-02-24T18:35:38.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc1f69a0 con 0x7f53cc15b860 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 10 client.4600.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 10 monclient: _renew_subs 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1f6f60 con 0x7f53cc15b860 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53d3474640 1 librados: init done 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c8010070 con 0x7f53cc15b860 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c800c990 con 0x7f53cc15b860 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53b9ffb640 10 monclient: dump: 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.196 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.183+0000 7f53bb7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.184+0000 7f53d3474640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.184+0000 7f53d3474640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.184+0000 7f53d3474640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.184+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53cc1594f0 con 0x7f53cc15b860 2026-02-24T18:35:38.226 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 10 monclient: monmap: 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.226152+0000 2026-02-24T18:35:38.227 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.226152+0000 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 1 Processor -- start 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 1 -- start start 2026-02-24T18:35:38.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.225+0000 7fdab99e3640 10 monclient: init 2026-02-24T18:35:38.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient: _add_conns ranks=[4,5,2,1,0,3] 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fdab4155fb0 0x7fdab415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-b con 0x7fdab4155fb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fdab415dfd0 0x7fdab415e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-c con 0x7fdab415dfd0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdab415f310 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab415e960 con 0x7fdab415f310 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdab415cce0 con 0x7fdab4155fb0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdab406f050 con 0x7fdab415dfd0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.226+0000 7fdab99e3640 10 monclient(hunting): authenticate will time out at 1481.907445s 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fdab415dfd0 0x7fdab415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fdab415dfd0 0x7fdab415e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44852/0 (socket says 10.20.193.18:44852) 2026-02-24T18:35:38.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 1 -- 10.20.193.18:0/3344242913 learned_addr learned my addr 10.20.193.18:0/3344242913 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 10 monclient(hunting): get_auth_request con 0x7fdab415dfd0 auth_method 0 2026-02-24T18:35:38.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.227+0000 7fdab27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.228+0000 7fdab27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.228+0000 7fdab27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.228+0000 7fdab27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 10 monclient(hunting): handle_auth_done global_id 4586 payload 274 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 1 -- 10.20.193.18:0/3344242913 >> v2:10.20.193.32:3300/0 conn(0x7fdab4155fb0 msgr2=0x7fdab415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 1 --2- 10.20.193.18:0/3344242913 >> v2:10.20.193.32:3300/0 conn(0x7fdab4155fb0 0x7fdab415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 1 -- 10.20.193.18:0/3344242913 >> v1:10.20.193.110:6789/0 conn(0x7fdab415f310 legacy=0x7fdab415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 1 monclient: found mon.noname-c 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.229+0000 7fdab27fc640 1 -- 10.20.193.18:0/3344242913 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab406f330 con 0x7fdab415dfd0 2026-02-24T18:35:38.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.231+0000 7fdab27fc640 1 --2- 10.20.193.18:0/3344242913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415dfd0 0x7fdab415e420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009970 tx=0x7fda9c02f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40bfce105de509a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 1 -- 10.20.193.18:0/3344242913 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c03c070 con 0x7fdab415dfd0 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: dump: 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.235 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 1 -- 10.20.193.18:0/3344242913 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c02fcd0 con 0x7fdab415dfd0 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 1 -- 10.20.193.18:0/3344242913 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c037660 con 0x7fdab415dfd0 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 5 monclient: authenticate success, global_id 4586 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 4 set_mon_vals no callback set 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab1ffb640 10 monclient: dump: 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 10 monclient: shutdown 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 -- 10.20.193.18:0/3344242913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415dfd0 msgr2=0x7fdab415e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 --2- 10.20.193.18:0/3344242913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415dfd0 0x7fdab415e420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009970 tx=0x7fda9c02f000 comp rx=0 tx=0).stop 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 -- 10.20.193.18:0/3344242913 shutdown_connections 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 --2- 10.20.193.18:0/3344242913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415dfd0 0x7fdab415e420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 --2- 10.20.193.18:0/3344242913 >> v2:10.20.193.32:3300/0 conn(0x7fdab4155fb0 0x7fdab415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 -- 10.20.193.18:0/3344242913 >> 10.20.193.18:0/3344242913 conn(0x7fdab406c820 msgr2=0x7fdab406cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 -- 10.20.193.18:0/3344242913 shutdown_connections 2026-02-24T18:35:38.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.233+0000 7fdab99e3640 1 -- 10.20.193.18:0/3344242913 wait complete. 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 10 monclient: monmap: 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 librados: starting msgr at 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 librados: starting objecter 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 Processor -- start 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 -- start start 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 librados: setting wanted keys 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 1 librados: calling monclient init 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.234+0000 7fdab99e3640 10 monclient: init 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdab4155fb0 0x7fdab41f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-a con 0x7fdab4155fb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdab415dfd0 0x7fdab40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-b con 0x7fdab415dfd0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): picked mon.noname-c con 0x7fdab415f310 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdab415d7d0 con 0x7fdab4155fb0 2026-02-24T18:35:38.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdab415d4f0 con 0x7fdab415dfd0 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdab415dab0 con 0x7fdab415f310 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab99e3640 10 monclient(hunting): authenticate will time out at 1481.916293s 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44856/0 (socket says 10.20.193.18:44856) 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 1 -- 10.20.193.18:0/1569765232 learned_addr learned my addr 10.20.193.18:0/1569765232 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 10 monclient(hunting): get_auth_request con 0x7fdab415f310 auth_method 0 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.235+0000 7fdab37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.236+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.236+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.236+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 10 monclient(hunting): handle_auth_done global_id 4592 payload 824 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 -- 10.20.193.18:0/1569765232 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdab415dfd0 msgr2=0x7fdab40cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdab415dfd0 0x7fdab40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 -- 10.20.193.18:0/1569765232 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdab4155fb0 msgr2=0x7fdab41f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdab4155fb0 0x7fdab41f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.237+0000 7fdab37fe640 1 -- 10.20.193.18:0/1569765232 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40c8d40 con 0x7fdab415f310 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.238+0000 7fdab37fe640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400cba0 tx=0x7fdaa400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.238+0000 7fda8bfff640 10 client.?.objecter ms_handle_connect 0x7fdab415f310 2026-02-24T18:35:38.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.238+0000 7fda8bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400ad50 con 0x7fdab415f310 2026-02-24T18:35:38.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 10 monclient: dump: 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fda8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fdab99e3640 5 monclient: authenticate success, global_id 4592 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fdab99e3640 10 monclient: _renew_subs 2026-02-24T18:35:38.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fdab99e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.239+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab41f4000 con 0x7fdab415f310 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 client.4592.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 monclient: _renew_subs 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41f45c0 con 0x7fdab415f310 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 1 librados: init done 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa4003a70 con 0x7fdab415f310 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab17fa640 4 set_mon_vals no callback set 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40050d0 con 0x7fdab415f310 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fda8bfff640 10 monclient: dump: 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.246 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.240+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab414ddc0 con 0x7fdab415f310 2026-02-24T18:35:38.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.250+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f27a0014a10 con 0x7f27ac15f8d0 2026-02-24T18:35:38.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.251+0000 7f2787fff640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 0x7f277c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.251+0000 7f27ab7fe640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 0x7f277c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.251+0000 7f27ab7fe640 10 monclient: get_auth_request con 0x7f277c078e00 auth_method 0 2026-02-24T18:35:38.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.251+0000 7f27ab7fe640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 0x7f277c07b2c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2798039000 tx=0x7f2798033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.251+0000 7f2787fff640 10 client.4629.objecter ms_handle_connect 0x7f277c078e00 2026-02-24T18:35:38.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.252+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fdaa4005270 con 0x7fdab415f310 2026-02-24T18:35:38.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.252+0000 7fda8bfff640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 0x7fda8407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.252+0000 7fdab2ffd640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 0x7fda8407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.252+0000 7fdab2ffd640 10 monclient: get_auth_request con 0x7fda84078e00 auth_method 0 2026-02-24T18:35:38.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fdab2ffd640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 0x7fda8407b2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80098a0 tx=0x7fdaa8006d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 10 client.4592.objecter ms_handle_connect 0x7fda84078e00 2026-02-24T18:35:38.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fdaa4098180 con 0x7fdab415f310 2026-02-24T18:35:38.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 10 client.4592.objecter ms_dispatch2 0x7fdab4000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 3 client.4592.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 3 client.4592.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.253+0000 7fda8bfff640 20 client.4592.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f27a005ef60 con 0x7f27ac15f8d0 2026-02-24T18:35:38.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 10 client.4629.objecter ms_dispatch2 0x7f27ac000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 3 client.4629.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 3 client.4629.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 20 client.4629.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f27a0022030 con 0x7f27ac15f8d0 2026-02-24T18:35:38.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.259+0000 7f2787fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.265+0000 7fe588dfe640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.265+0000 7fe588dfe640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.265+0000 7fe588dfe640 10 monclient: monmap: 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.266542+0000 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.266542+0000 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.267 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.268 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.268 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.268 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.268 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.268 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 Processor -- start 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 -- start start 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient: init 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient: _add_conns ranks=[0,4,2,1,3,5] 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe5841527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe584152bb0 0x7fe5840c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-b con 0x7fe584152bb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe5840c5fc0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58406e300 con 0x7fe5841527d0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58406e5e0 con 0x7fe5840c5fc0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe58406f1f0 con 0x7fe584152bb0 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.266+0000 7fe588dfe640 10 monclient(hunting): authenticate will time out at 1481.947747s 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.267+0000 7fe582575640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fe5841527d0 0x7fe58406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58806/0 (socket says 10.20.193.18:58806) 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.267+0000 7fe582575640 1 -- 10.20.193.18:0/3221668673 learned_addr learned my addr 10.20.193.18:0/3221668673 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644817637 0 0) 0x7fe58406e300 con 0x7fe5841527d0 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 10 monclient(hunting): _negotiate 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 10 monclient(hunting): my global_id is 4632 2026-02-24T18:35:38.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.268+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe570003620 con 0x7fe5841527d0 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 296994906 0 0) 0x7fe570003620 con 0x7fe5841527d0 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 >> v2:10.20.193.32:3300/0 conn(0x7fe584152bb0 msgr2=0x7fe5840c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 --2- 10.20.193.18:0/3221668673 >> v2:10.20.193.32:3300/0 conn(0x7fe584152bb0 0x7fe5840c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 >> v1:10.20.193.110:6789/0 conn(0x7fe5840c5fc0 legacy=0x7fe5840c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58406e8c0 con 0x7fe5841527d0 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7fe578002f00 con 0x7fe5841527d0 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: dump: 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.274 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 >> v1:10.20.193.18:6789/0 conn(0x7fe5841527d0 legacy=0x7fe58406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe570003ba0 0x7fe570003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): picked mon.c con 0x7fe570003ba0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe570004900 0x7fe570004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): picked mon.b con 0x7fe570004900 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe57000bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): picked mon.a con 0x7fe570005510 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe58406f1f0 con 0x7fe570005510 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe57000c450 con 0x7fe570004900 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe57000c650 con 0x7fe570003ba0 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.270+0000 7fe581573640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.271+0000 7fe582d76640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe57000bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.271+0000 7fe582d76640 10 monclient(hunting): get_auth_request con 0x7fe570005510 auth_method 0 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.271+0000 7fe582d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.271+0000 7fe582d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.271+0000 7fe582d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient(hunting): handle_auth_done global_id 4632 payload 293 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 -- 10.20.193.18:0/3221668673 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe570003ba0 msgr2=0x7fe570003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe570003ba0 0x7fe570003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 -- 10.20.193.18:0/3221668673 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe570004900 msgr2=0x7fe570004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe570004900 0x7fe570004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 monclient: found mon.a 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 1 -- 10.20.193.18:0/3221668673 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570003620 con 0x7fe570005510 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe582d76640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe588dfe640 5 monclient: authenticate success, global_id 4632 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.272+0000 7fe588dfe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.273+0000 7fe582d76640 1 -- 10.20.193.18:0/3221668673 reap_dead start 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.273+0000 7fe582d76640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe57000bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe574003a20 tx=0x7fe57402f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55062cf0bd2163 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740035f0 con 0x7fe570005510 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: dump: 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.280 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574003a70 con 0x7fe570005510 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe588dfe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 1 -- 10.20.193.18:0/3221668673 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57402fe60 con 0x7fe570005510 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe588dfe640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe588dfe640 4 set_mon_vals no callback set 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.275+0000 7fe581573640 10 monclient: dump: 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.281 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 10 monclient: shutdown 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 -- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 msgr2=0x7fe57000bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe57000bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe574003a20 tx=0x7fe57402f3e0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 -- 10.20.193.18:0/3221668673 shutdown_connections 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 --2- 10.20.193.18:0/3221668673 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe57000bb70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 -- 10.20.193.18:0/3221668673 >> 10.20.193.18:0/3221668673 conn(0x7fe5840cd780 msgr2=0x7fe5840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 -- 10.20.193.18:0/3221668673 shutdown_connections 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 1 -- 10.20.193.18:0/3221668673 wait complete. 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.276+0000 7fe588dfe640 10 monclient: monmap: 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.282 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 librados: starting msgr at 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 librados: starting objecter 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 Processor -- start 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 -- start start 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 librados: setting wanted keys 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 librados: calling monclient init 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient: init 2026-02-24T18:35:38.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-a con 0x7fe570005510 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe584146720 0x7fe5840ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-b con 0x7fe584146720 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5840bcf70 0x7fe5840bab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): picked mon.noname-c con 0x7fe5840bcf70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe58406ee80 con 0x7fe570005510 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840c6470 con 0x7fe584146720 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840c5a80 con 0x7fe5840bcf70 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.277+0000 7fe588dfe640 10 monclient(hunting): authenticate will time out at 1481.958738s 2026-02-24T18:35:38.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42150/0 (socket says 10.20.193.18:42150) 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 1 -- 10.20.193.18:0/239996164 learned_addr learned my addr 10.20.193.18:0/239996164 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): get_auth_request con 0x7fe570005510 auth_method 0 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.279+0000 7fe582575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 10 monclient(hunting): handle_auth_done global_id 4635 payload 824 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 -- 10.20.193.18:0/239996164 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5840bcf70 msgr2=0x7fe5840bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5840bcf70 0x7fe5840bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 -- 10.20.193.18:0/239996164 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe584146720 msgr2=0x7fe5840ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe584146720 0x7fe5840ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 monclient: found mon.noname-a 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.281+0000 7fe582575640 1 -- 10.20.193.18:0/239996164 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5841f3670 con 0x7fe570005510 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.282+0000 7fe582575640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe57800caa0 tx=0x7fe57800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.282+0000 7fe56affd640 10 client.?.objecter ms_handle_connect 0x7fe570005510 2026-02-24T18:35:38.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.282+0000 7fe56affd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578004cf0 con 0x7fe570005510 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: dump: 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe578007110 con 0x7fe570005510 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780074e0 con 0x7fe570005510 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 5 monclient: authenticate success, global_id 4635 2026-02-24T18:35:38.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe580d72640 4 set_mon_vals no callback set 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _renew_subs 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841f3950 con 0x7fe570005510 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe56affd640 10 monclient: dump: 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 client.4635.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _renew_subs 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5841f3f10 con 0x7fe570005510 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 1 librados: init done 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.284+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58406da60 con 0x7fe570005510 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.286+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f53c801d030 con 0x7f53cc15b860 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.286+0000 7f53b9ffb640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 0x7f539c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.286+0000 7f53d11e9640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 0x7f539c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.286+0000 7f53d11e9640 10 monclient: get_auth_request con 0x7f539c078e00 auth_method 0 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.287+0000 7f53b9ffb640 10 client.4600.objecter ms_handle_connect 0x7f539c078e00 2026-02-24T18:35:38.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.287+0000 7f53d11e9640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 0x7f539c07b2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53bc006b90 tx=0x7f53bc006b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.288+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f53c80970a0 con 0x7f53cc15b860 2026-02-24T18:35:38.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.288+0000 7f53b9ffb640 10 client.4600.objecter ms_dispatch2 0x7f53cc000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.288+0000 7f53b9ffb640 3 client.4600.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.288+0000 7f53b9ffb640 3 client.4600.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.288+0000 7f53b9ffb640 20 client.4600.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.295+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f53c8061db0 con 0x7f53cc15b860 2026-02-24T18:35:38.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.295+0000 7f53b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.295+0000 7f53b9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.296+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe57800a5e0 con 0x7fe570005510 2026-02-24T18:35:38.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.296+0000 7fe56affd640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 0x7fe55007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.296+0000 7fe581d74640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 0x7fe55007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.296+0000 7fe581d74640 10 monclient: get_auth_request con 0x7fe550078e30 auth_method 0 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 10 monclient: monmap: 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.297975+0000 2026-02-24T18:35:38.298 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.297975+0000 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 1 Processor -- start 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 1 -- start start 2026-02-24T18:35:38.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7f996a0a9640 10 monclient: init 2026-02-24T18:35:38.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7fe581d74640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 0x7fe55007b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5841f1230 tx=0x7fe56c009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.297+0000 7fe56affd640 10 client.4635.objecter ms_handle_connect 0x7fe550078e30 2026-02-24T18:35:38.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient: _add_conns ranks=[0,4,1,2,3,5] 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f99641527f0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9964073530 0x7f996415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-b con 0x7f9964073530 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f996415f900 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99640739a0 con 0x7f99641527f0 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996415fd90 con 0x7f996415f900 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9964074b10 con 0x7f9964073530 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7f996a0a9640 10 monclient(hunting): authenticate will time out at 1481.979249s 2026-02-24T18:35:38.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fe578007680 con 0x7fe570005510 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7fe56affd640 10 client.4635.objecter ms_dispatch2 0x7fe584000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7fe56affd640 3 client.4635.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.298+0000 7fe56affd640 3 client.4635.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.299+0000 7fe56affd640 20 client.4635.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.299+0000 7f9963fff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f996415f900 0x7f99640ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51390/0 (socket says 10.20.193.18:51390) 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.299+0000 7f9963fff640 1 -- 10.20.193.18:0/3873448941 learned_addr learned my addr 10.20.193.18:0/3873448941 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 1 --2- 10.20.193.18:0/3873448941 >> v2:10.20.193.32:3300/0 conn(0x7f9964073530 0x7f996415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): get_auth_request con 0x7f9964073530 auth_method 0 2026-02-24T18:35:38.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.300+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 10 monclient(hunting): handle_auth_done global_id 4603 payload 274 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 1 -- 10.20.193.18:0/3873448941 >> v1:10.20.193.32:6789/0 conn(0x7f996415f900 legacy=0x7f99640ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 1 -- 10.20.193.18:0/3873448941 >> v1:10.20.193.18:6789/0 conn(0x7f99641527f0 legacy=0x7f996415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 1 monclient: found mon.noname-b 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.302+0000 7f9962ffd640 1 -- 10.20.193.18:0/3873448941 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996415d9d0 con 0x7f9964073530 2026-02-24T18:35:38.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.303+0000 7f9962ffd640 1 --2- 10.20.193.18:0/3873448941 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9964073530 0x7f996415f3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99540098a0 tx=0x7f995402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448fa2585199425c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 1 -- 10.20.193.18:0/3873448941 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995403c070 con 0x7f9964073530 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: dump: 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.307 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 1 -- 10.20.193.18:0/3873448941 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995402fad0 con 0x7f9964073530 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 5 monclient: authenticate success, global_id 4603 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 1 -- 10.20.193.18:0/3873448941 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954037440 con 0x7f9964073530 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f99627fc640 10 monclient: dump: 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.308 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 4 set_mon_vals no callback set 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 10 monclient: shutdown 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 -- 10.20.193.18:0/3873448941 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9964073530 msgr2=0x7f996415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 --2- 10.20.193.18:0/3873448941 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9964073530 0x7f996415f3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99540098a0 tx=0x7f995402ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 -- 10.20.193.18:0/3873448941 shutdown_connections 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 --2- 10.20.193.18:0/3873448941 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9964073530 0x7f996415f3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 -- 10.20.193.18:0/3873448941 >> 10.20.193.18:0/3873448941 conn(0x7f99640ce730 msgr2=0x7f99640ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 -- 10.20.193.18:0/3873448941 shutdown_connections 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.305+0000 7f996a0a9640 1 -- 10.20.193.18:0/3873448941 wait complete. 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient: monmap: 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.309 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.310 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 librados: starting msgr at 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 librados: starting objecter 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 Processor -- start 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 -- start start 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 librados: setting wanted keys 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 librados: calling monclient init 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient: init 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9964073530 0x7f99641f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-c con 0x7f9964073530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-b con 0x7f99641527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f996415f900 0x7f9964148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): picked mon.noname-a con 0x7f996415f900 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99640741e0 con 0x7f996415f900 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9964073f00 con 0x7f99641527f0 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99640744c0 con 0x7f9964073530 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.306+0000 7f996a0a9640 10 monclient(hunting): authenticate will time out at 1481.987779s 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53942/0 (socket says 10.20.193.18:53942) 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 1 -- 10.20.193.18:0/1200668010 learned_addr learned my addr 10.20.193.18:0/1200668010 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): get_auth_request con 0x7f99641527f0 auth_method 0 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.307+0000 7f9962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 10 monclient(hunting): handle_auth_done global_id 4606 payload 824 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 -- 10.20.193.18:0/1200668010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9964073530 msgr2=0x7f99641f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9964073530 0x7f99641f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 -- 10.20.193.18:0/1200668010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f996415f900 msgr2=0x7f9964148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f996415f900 0x7f9964148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 monclient: found mon.noname-b 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.309+0000 7f9962ffd640 1 -- 10.20.193.18:0/1200668010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99641e7f70 con 0x7f99641527f0 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.310+0000 7f9962ffd640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9954009130 tx=0x7f99540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.310+0000 7f993ffff640 10 client.?.objecter ms_handle_connect 0x7f99641527f0 2026-02-24T18:35:38.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.310+0000 7f993ffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995403c040 con 0x7f99641527f0 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 10 monclient: dump: 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.315 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f993ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 5 monclient: authenticate success, global_id 4606 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _renew_subs 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641e8250 con 0x7f99641527f0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 client.4606.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _renew_subs 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641e8c60 con 0x7f99641527f0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 1 librados: init done 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.311+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9964077010 con 0x7f99641527f0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995402fad0 con 0x7f99641527f0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995403f880 con 0x7f99641527f0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f993ffff640 10 monclient: dump: 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.312+0000 7f9961ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.345+0000 7f27b0ea3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.345+0000 7f27b0ea3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.345+0000 7f27b0ea3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.345+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f27ac0cb4e0 con 0x7f27ac15f8d0 2026-02-24T18:35:38.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.354+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdab414ddc0 con 0x7fdab415f310 2026-02-24T18:35:38.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.354+0000 7fda8bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.354+0000 7fda8bfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.381+0000 7f53d3474640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:38.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.381+0000 7f53d3474640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:38.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.381+0000 7f53d3474640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.381+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f53cc0c10d0 con 0x7f53cc15b860 2026-02-24T18:35:38.388 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-24T18:35:38.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.402+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe5780627d0 con 0x7fe570005510 2026-02-24T18:35:38.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.402+0000 7fe56affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.402+0000 7fe56affd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.409+0000 7f2787fff640 1 -- 10.20.193.18:0/1937639606 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27a0010040 con 0x7f27ac15f8d0 2026-02-24T18:35:38.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.409+0000 7f2787fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-24T18:35:38.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.409+0000 7f2787fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.411 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:38.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 10 librados: watch_flush enter 2026-02-24T18:35:38.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 10 librados: watch_flush exit 2026-02-24T18:35:38.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 20 client.4629.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 10 client.4629.objecter successfully canceled tick 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 msgr2=0x7f277c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 0x7f277c07b2c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2798039000 tx=0x7f2798033000 comp rx=0 tx=0).stop 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 10 monclient: shutdown 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 msgr2=0x7f27ac156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f27a000efc0 tx=0x7f27a000c490 comp rx=0 tx=0).stop 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 shutdown_connections 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f277c078e00 0x7f277c07b2c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f27ac159b40 0x7f27ac14e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f27ac15f8d0 0x7f27ac156cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 --2- 10.20.193.18:0/1937639606 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f27ac152510 0x7f27ac1f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 >> 10.20.193.18:0/1937639606 conn(0x7f27ac0ce330 msgr2=0x7f27ac15bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 shutdown_connections 2026-02-24T18:35:38.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 -- 10.20.193.18:0/1937639606 wait complete. 2026-02-24T18:35:38.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.411+0000 7f27b0ea3640 1 librados: shutdown 2026-02-24T18:35:38.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.416+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f995403fa20 con 0x7f99641527f0 2026-02-24T18:35:38.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.416+0000 7f993ffff640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 0x7f993407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.416+0000 7f99637fe640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 0x7f993407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.417+0000 7f99637fe640 10 monclient: get_auth_request con 0x7f9934078e30 auth_method 0 2026-02-24T18:35:38.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.417+0000 7f993ffff640 10 client.4606.objecter ms_handle_connect 0x7f9934078e30 2026-02-24T18:35:38.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.417+0000 7f99637fe640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 0x7f993407b2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9950006fd0 tx=0x7f9950006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.418+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7f995403fd40 con 0x7f99641527f0 2026-02-24T18:35:38.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.418+0000 7f993ffff640 10 client.4606.objecter ms_dispatch2 0x7f9964000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.418+0000 7f993ffff640 3 client.4606.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.418+0000 7f993ffff640 3 client.4606.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.420 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.418+0000 7f993ffff640 20 client.4606.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.0 2026-02-24T18:35:38.423 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.426+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9964077010 con 0x7f99641527f0 2026-02-24T18:35:38.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.426+0000 7f993ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.426+0000 7f993ffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.429+0000 7f53b9ffb640 1 -- 10.20.193.18:0/1292392506 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f53c80616b0 con 0x7f53cc15b860 2026-02-24T18:35:38.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.429+0000 7f53b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-24T18:35:38.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.429+0000 7f53b9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.431 INFO:teuthology.orchestra.run.trial018.stdout:98784247812 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 10 librados: watch_flush enter 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 10 librados: watch_flush exit 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 20 client.4600.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 10 client.4600.objecter successfully canceled tick 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 msgr2=0x7f539c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 0x7f539c07b2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53bc006b90 tx=0x7f53bc006b20 comp rx=0 tx=0).stop 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 10 monclient: shutdown 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 msgr2=0x7f53cc0c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b9c0 tx=0x7f53c800be90 comp rx=0 tx=0).stop 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 shutdown_connections 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f539c078e00 0x7f539c07b2c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f53cc15b860 0x7f53cc0c1370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53cc0740a0 0x7f53cc1e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 --2- 10.20.193.18:0/1292392506 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f53cc072480 0x7f53cc151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 >> 10.20.193.18:0/1292392506 conn(0x7f53cc0cd780 msgr2=0x7f53cc0c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 shutdown_connections 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 -- 10.20.193.18:0/1292392506 wait complete. 2026-02-24T18:35:38.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.431+0000 7f53d3474640 1 librados: shutdown 2026-02-24T18:35:38.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.7 2026-02-24T18:35:38.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.441+0000 7fdab99e3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:38.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.441+0000 7fdab99e3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:38.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.441+0000 7fdab99e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:38.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.441+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdab4156390 con 0x7fdab415f310 2026-02-24T18:35:38.444 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 10 monclient: monmap: 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.458221+0000 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.458221+0000 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.459 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.460 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 1 Processor -- start 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 1 -- start start 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.457+0000 7fa735e3b640 10 monclient: init 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient: _add_conns ranks=[3,2,5,4,0,1] 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa7300c8fe0 0x7fa7300c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-a con 0x7fa7300c8fe0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:38.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa73006d930 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa73006dd80 0x7fa730159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-c con 0x7fa73006dd80 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa730159790 con 0x7fa73006d930 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa73006e290 con 0x7fa7300c8fe0 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa73006e570 con 0x7fa73006dd80 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa735e3b640 10 monclient(hunting): authenticate will time out at 1482.139412s 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa7300c8fe0 0x7fa7300c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa7300c8fe0 0x7fa7300c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42156/0 (socket says 10.20.193.18:42156) 2026-02-24T18:35:38.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 1 -- 10.20.193.18:0/3488582137 learned_addr learned my addr 10.20.193.18:0/3488582137 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 10 monclient(hunting): get_auth_request con 0x7fa7300c8fe0 auth_method 0 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.458+0000 7fa734e39640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.459+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.459+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.459+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 10 monclient(hunting): handle_auth_done global_id 4638 payload 274 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 1 -- 10.20.193.18:0/3488582137 >> v2:10.20.193.110:3300/0 conn(0x7fa73006dd80 msgr2=0x7fa730159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 1 --2- 10.20.193.18:0/3488582137 >> v2:10.20.193.110:3300/0 conn(0x7fa73006dd80 0x7fa730159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 1 -- 10.20.193.18:0/3488582137 >> v1:10.20.193.110:6789/0 conn(0x7fa73006d930 legacy=0x7fa73015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 1 monclient: found mon.noname-a 2026-02-24T18:35:38.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.461+0000 7fa734e39640 1 -- 10.20.193.18:0/3488582137 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73006e850 con 0x7fa7300c8fe0 2026-02-24T18:35:38.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.465+0000 7fa734e39640 1 --2- 10.20.193.18:0/3488582137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa7300c8fe0 0x7fa7300c93c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa720009970 tx=0x7fa72002efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573b9fabb9f28bcc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 1 -- 10.20.193.18:0/3488582137 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72003c070 con 0x7fa7300c8fe0 2026-02-24T18:35:38.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: dump: 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 1 -- 10.20.193.18:0/3488582137 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72002fbe0 con 0x7fa7300c8fe0 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 5 monclient: authenticate success, global_id 4638 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 1 -- 10.20.193.18:0/3488582137 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720037440 con 0x7fa7300c8fe0 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 4 set_mon_vals no callback set 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa7277fe640 10 monclient: dump: 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 10 monclient: shutdown 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 -- 10.20.193.18:0/3488582137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa7300c8fe0 msgr2=0x7fa7300c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 --2- 10.20.193.18:0/3488582137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa7300c8fe0 0x7fa7300c93c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa720009970 tx=0x7fa72002efb0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 -- 10.20.193.18:0/3488582137 shutdown_connections 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 --2- 10.20.193.18:0/3488582137 >> v2:10.20.193.110:3300/0 conn(0x7fa73006dd80 0x7fa730159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 --2- 10.20.193.18:0/3488582137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa7300c8fe0 0x7fa7300c93c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 -- 10.20.193.18:0/3488582137 >> 10.20.193.18:0/3488582137 conn(0x7fa7300cd480 msgr2=0x7fa7300cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 -- 10.20.193.18:0/3488582137 shutdown_connections 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.467+0000 7fa735e3b640 1 -- 10.20.193.18:0/3488582137 wait complete. 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient: monmap: 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.470 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 librados: starting msgr at 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 librados: starting objecter 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 Processor -- start 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 -- start start 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 librados: setting wanted keys 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 librados: calling monclient init 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient: init 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-b con 0x7fa73006d930 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa73006dd80 0x7fa730072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-a con 0x7fa73006dd80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa7300c8fe0 0x7fa730072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): picked mon.noname-c con 0x7fa7300c8fe0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa73006f0f0 con 0x7fa73006dd80 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa73006ee10 con 0x7fa73006d930 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa73015cc10 con 0x7fa7300c8fe0 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.468+0000 7fa735e3b640 10 monclient(hunting): authenticate will time out at 1482.150004s 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa727fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa73006dd80 0x7fa730072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53958/0 (socket says 10.20.193.18:53958) 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 1 -- 10.20.193.18:0/639182869 learned_addr learned my addr 10.20.193.18:0/639182869 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): get_auth_request con 0x7fa73006d930 auth_method 0 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa727fff640 10 monclient(hunting): get_auth_request con 0x7fa73006dd80 auth_method 0 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa727fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa727fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.469+0000 7fa734e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fa727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fa727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fa727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fda8bfff640 1 -- 10.20.193.18:0/1569765232 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdaa401a360 con 0x7fdab415f310 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fda8bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.470+0000 7fda8bfff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stdout:90194313220 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 10 monclient(hunting): handle_auth_done global_id 4609 payload 824 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 -- 10.20.193.18:0/639182869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa7300c8fe0 msgr2=0x7fa730072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa7300c8fe0 0x7fa730072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 -- 10.20.193.18:0/639182869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa73006dd80 msgr2=0x7fa730072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa73006dd80 0x7fa730072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 monclient: found mon.noname-b 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.471+0000 7fa734e39640 1 -- 10.20.193.18:0/639182869 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7301f3ed0 con 0x7fa73006d930 2026-02-24T18:35:38.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.472+0000 7fa734e39640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa720009970 tx=0x7fa7200047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.472+0000 7fa7257fa640 10 client.?.objecter ms_handle_connect 0x7fa73006d930 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.472+0000 7fa7257fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.472+0000 7fdab99e3640 10 librados: watch_flush enter 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 10 librados: watch_flush exit 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 20 client.4592.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 10 client.4592.objecter successfully canceled tick 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 msgr2=0x7fda8407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 0x7fda8407b2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80098a0 tx=0x7fdaa8006d90 comp rx=0 tx=0).stop 2026-02-24T18:35:38.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 10 monclient: shutdown 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 msgr2=0x7fdab41535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400cba0 tx=0x7fdaa400a430 comp rx=0 tx=0).stop 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 shutdown_connections 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fda84078e00 0x7fda8407b2c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdab415dfd0 0x7fdab40cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdab415f310 0x7fdab41535b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 --2- 10.20.193.18:0/1569765232 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdab4155fb0 0x7fdab41f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 >> 10.20.193.18:0/1569765232 conn(0x7fdab406c820 msgr2=0x7fdab40c7090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 shutdown_connections 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 -- 10.20.193.18:0/1569765232 wait complete. 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.473+0000 7fdab99e3640 1 librados: shutdown 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72003c040 con 0x7fa73006d930 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: dump: 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa720004230 con 0x7fa73006d930 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 5 monclient: authenticate success, global_id 4609 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _renew_subs 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7301f41b0 con 0x7fa73006d930 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720003710 con 0x7fa73006d930 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa7257fa640 10 monclient: dump: 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.476 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa726ffd640 4 set_mon_vals no callback set 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 client.4609.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _renew_subs 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7301f4770 con 0x7fa73006d930 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 1 librados: init done 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.474+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73015cf00 con 0x7fa73006d930 2026-02-24T18:35:38.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313220 for osd.6 2026-02-24T18:35:38.483 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.485+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa72003f050 con 0x7fa73006d930 2026-02-24T18:35:38.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.485+0000 7fa7257fa640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 0x7fa70007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.486+0000 7fa727fff640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 0x7fa70007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.486+0000 7fa727fff640 10 monclient: get_auth_request con 0x7fa700078e80 auth_method 0 2026-02-24T18:35:38.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.486+0000 7fa7257fa640 10 client.4609.objecter ms_handle_connect 0x7fa700078e80 2026-02-24T18:35:38.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.486+0000 7fa727fff640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 0x7fa70007b340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7300c6ac0 tx=0x7fa7180091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.487+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fa72002fb80 con 0x7fa73006d930 2026-02-24T18:35:38.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.487+0000 7fa7257fa640 10 client.4609.objecter ms_dispatch2 0x7fa730000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.487+0000 7fa7257fa640 3 client.4609.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.487+0000 7fa7257fa640 3 client.4609.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.487+0000 7fa7257fa640 20 client.4609.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.490+0000 7fe588dfe640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:38.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.490+0000 7fe588dfe640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:38.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.490+0000 7fe588dfe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.490+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe5840c77d0 con 0x7fe570005510 2026-02-24T18:35:38.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.493+0000 7fe56affd640 1 -- 10.20.193.18:0/239996164 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5780620d0 con 0x7fe570005510 2026-02-24T18:35:38.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.493+0000 7fe56affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-24T18:35:38.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.493+0000 7fe56affd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.494 INFO:teuthology.orchestra.run.trial018.stdout:98784247813 2026-02-24T18:35:38.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 10 librados: watch_flush enter 2026-02-24T18:35:38.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 10 librados: watch_flush exit 2026-02-24T18:35:38.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 20 client.4635.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 10 client.4635.objecter successfully canceled tick 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 msgr2=0x7fe55007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 0x7fe55007b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5841f1230 tx=0x7fe56c009250 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 10 monclient: shutdown 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 msgr2=0x7fe5841461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe57800caa0 tx=0x7fe57800cf70 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 shutdown_connections 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe550078e30 0x7fe55007b2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe584146720 0x7fe5840ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5840bcf70 0x7fe5840bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 --2- 10.20.193.18:0/239996164 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe570005510 0x7fe5841461e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 >> 10.20.193.18:0/239996164 conn(0x7fe5840cd780 msgr2=0x7fe58414fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 shutdown_connections 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 -- 10.20.193.18:0/239996164 wait complete. 2026-02-24T18:35:38.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.495+0000 7fe588dfe640 1 librados: shutdown 2026-02-24T18:35:38.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.5 2026-02-24T18:35:38.505 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.515+0000 7f996a0a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:38.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.515+0000 7f996a0a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:38.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.515+0000 7f996a0a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.515+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f99640cbee0 con 0x7f99641527f0 2026-02-24T18:35:38.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.584+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa73015cf00 con 0x7fa73006d930 2026-02-24T18:35:38.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.584+0000 7fa7257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.584+0000 7fa7257fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.588+0000 7f993ffff640 1 -- 10.20.193.18:0/1200668010 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99640cbee0 con 0x7f99641527f0 2026-02-24T18:35:38.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.588+0000 7f993ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-24T18:35:38.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.588+0000 7f993ffff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.590 INFO:teuthology.orchestra.run.trial018.stdout:98784247811 2026-02-24T18:35:38.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 10 librados: watch_flush enter 2026-02-24T18:35:38.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 10 librados: watch_flush exit 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 20 client.4606.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 10 client.4606.objecter successfully canceled tick 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 msgr2=0x7f993407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 0x7f993407b2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9950006fd0 tx=0x7f9950006d40 comp rx=0 tx=0).stop 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 10 monclient: shutdown 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 msgr2=0x7f99641e9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9954009130 tx=0x7f99540047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 shutdown_connections 2026-02-24T18:35:38.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9934078e30 0x7f993407b2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99641527f0 0x7f99641e9740 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f996415f900 0x7f9964148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 --2- 10.20.193.18:0/1200668010 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9964073530 0x7f99641f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 >> 10.20.193.18:0/1200668010 conn(0x7f99640ce730 msgr2=0x7f996415d250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 shutdown_connections 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 -- 10.20.193.18:0/1200668010 wait complete. 2026-02-24T18:35:38.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.590+0000 7f996a0a9640 1 librados: shutdown 2026-02-24T18:35:38.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.3 2026-02-24T18:35:38.601 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.670+0000 7fa735e3b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:38.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.670+0000 7fa735e3b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:38.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.670+0000 7fa735e3b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:38.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.670+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa7300c93c0 con 0x7fa73006d930 2026-02-24T18:35:38.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa7257fa640 1 -- 10.20.193.18:0/639182869 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fa720044020 con 0x7fa73006d930 2026-02-24T18:35:38.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa7257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-24T18:35:38.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa7257fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:38.674 INFO:teuthology.orchestra.run.trial018.stdout:94489280517 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 10 librados: watch_flush enter 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 10 librados: watch_flush exit 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 20 client.4609.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 10 client.4609.objecter successfully canceled tick 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 msgr2=0x7fa70007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 0x7fa70007b340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7300c6ac0 tx=0x7fa7180091c0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 10 monclient: shutdown 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 msgr2=0x7fa73014dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa720009970 tx=0x7fa7200047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 shutdown_connections 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa700078e80 0x7fa70007b340 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa7300c8fe0 0x7fa730072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa73006dd80 0x7fa730072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 --2- 10.20.193.18:0/639182869 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa73006d930 0x7fa73014dbc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 >> 10.20.193.18:0/639182869 conn(0x7fa7300cd480 msgr2=0x7fa7301e79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 shutdown_connections 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 -- 10.20.193.18:0/639182869 wait complete. 2026-02-24T18:35:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.674+0000 7fa735e3b640 1 librados: shutdown 2026-02-24T18:35:38.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280517 for osd.11 2026-02-24T18:35:38.685 DEBUG:teuthology.parallel:result is None 2026-02-24T18:35:38.685 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T18:35:38.685 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:35:38.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.753+0000 7fc04a858640 10 monclient: get_monmap_and_config 2026-02-24T18:35:38.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.753+0000 7fc04a858640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.753+0000 7fc04a858640 10 monclient: monmap: 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:38.754458+0000 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:38.754458+0000 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:38.756 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 Processor -- start 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 -- start start 2026-02-24T18:35:38.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient: init 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient: _add_conns ranks=[2,3,0,1,5,4] 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc04406ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc0440c99a0 0x7fc044070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-a con 0x7fc0440c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc04415a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0440c7e90 con 0x7fc04415a5e0 2026-02-24T18:35:38.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0440c8170 con 0x7fc04406ff20 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc044159a40 con 0x7fc0440c99a0 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.754+0000 7fc04a858640 10 monclient(hunting): authenticate will time out at 1482.435683s 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc0440c99a0 0x7fc044070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc043fff640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fc04406ff20 0x7fc044159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58442/0 (socket says 10.20.193.18:58442) 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc043fff640 1 -- 10.20.193.18:0/281190395 learned_addr learned my addr 10.20.193.18:0/281190395 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): get_auth_request con 0x7fc0440c99a0 auth_method 0 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.755+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 1 -- 10.20.193.18:0/281190395 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374441491 0 0) 0x7fc0440c8170 con 0x7fc04406ff20 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 10 monclient(hunting): my global_id is 4595 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.756+0000 7fc042ffd640 1 -- 10.20.193.18:0/281190395 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc028003620 con 0x7fc04406ff20 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 10 monclient(hunting): handle_auth_done global_id 4644 payload 274 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 1 -- 10.20.193.18:0/281190395 >> v1:10.20.193.110:6789/0 conn(0x7fc04406ff20 legacy=0x7fc044159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 1 -- 10.20.193.18:0/281190395 >> v1:10.20.193.18:6789/0 conn(0x7fc04415a5e0 legacy=0x7fc04414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:38.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.757+0000 7fc0437fe640 1 -- 10.20.193.18:0/281190395 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0440c8450 con 0x7fc0440c99a0 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.759+0000 7fc0437fe640 1 --2- 10.20.193.18:0/281190395 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc044070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098d0 tx=0x7fc03402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdee2c5447a89e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 1 -- 10.20.193.18:0/281190395 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03403c070 con 0x7fc0440c99a0 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: dump: 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.770 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 1 -- 10.20.193.18:0/281190395 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03402fba0 con 0x7fc0440c99a0 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc04a858640 5 monclient: authenticate success, global_id 4644 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc04a858640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 1 -- 10.20.193.18:0/281190395 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034037440 con 0x7fc0440c99a0 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc042ffd640 10 monclient: dump: 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.772 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc04a858640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.760+0000 7fc04a858640 4 set_mon_vals no callback set 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 10 monclient: shutdown 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- 10.20.193.18:0/281190395 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 msgr2=0x7fc044070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 --2- 10.20.193.18:0/281190395 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc044070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098d0 tx=0x7fc03402eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- 10.20.193.18:0/281190395 shutdown_connections 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 --2- 10.20.193.18:0/281190395 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc044070300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- 10.20.193.18:0/281190395 >> 10.20.193.18:0/281190395 conn(0x7fc0440cd780 msgr2=0x7fc0440cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- 10.20.193.18:0/281190395 shutdown_connections 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- 10.20.193.18:0/281190395 wait complete. 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 10 monclient: build_initial_monmap 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 10 monclient: monmap: 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:38.774 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 librados: starting msgr at 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 librados: starting objecter 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 Processor -- start 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 -- start start 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 librados: setting wanted keys 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 1 librados: calling monclient init 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.761+0000 7fc04a858640 10 monclient: init 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:38.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc04406ff20 0x7fc04414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-c con 0x7fc04406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-a con 0x7fc0440c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc04415a5e0 0x7fc0440cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): picked mon.noname-b con 0x7fc04415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc044150520 con 0x7fc0440c99a0 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440c8a10 con 0x7fc04415a5e0 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441506a0 con 0x7fc04406ff20 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:38.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc04a858640 10 monclient(hunting): authenticate will time out at 1482.443294s 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42180/0 (socket says 10.20.193.18:42180) 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 1 -- 10.20.193.18:0/2017513827 learned_addr learned my addr 10.20.193.18:0/2017513827 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): get_auth_request con 0x7fc0440c99a0 auth_method 0 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc043fff640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc04406ff20 0x7fc04414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc048dce640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc04415a5e0 0x7fc0440cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc0437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc048dce640 10 monclient(hunting): get_auth_request con 0x7fc04415a5e0 auth_method 0 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc048dce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc048dce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc048dce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc043fff640 10 monclient(hunting): get_auth_request con 0x7fc04406ff20 auth_method 0 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc043fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.762+0000 7fc043fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc048dce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc048dce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc048dce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:38.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.763+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 10 monclient(hunting): handle_auth_done global_id 4650 payload 824 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 -- 10.20.193.18:0/2017513827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc04406ff20 msgr2=0x7fc04414c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc04406ff20 0x7fc04414c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 -- 10.20.193.18:0/2017513827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc04415a5e0 msgr2=0x7fc0440cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc04415a5e0 0x7fc0440cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.764+0000 7fc0437fe640 1 -- 10.20.193.18:0/2017513827 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441e50a0 con 0x7fc0440c99a0 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.765+0000 7fc0437fe640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc03403a040 tx=0x7fc0340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.765+0000 7fc040ff9640 10 client.?.objecter ms_handle_connect 0x7fc0440c99a0 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.765+0000 7fc040ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03403c040 con 0x7fc0440c99a0 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: dump: 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.780 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034004230 con 0x7fc0440c99a0 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 5 monclient: authenticate success, global_id 4650 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 10 monclient: _renew_subs 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441e5380 con 0x7fc0440c99a0 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034004620 con 0x7fc0440c99a0 2026-02-24T18:35:38.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc0427fc640 4 set_mon_vals no callback set 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc040ff9640 10 monclient: dump: 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:38.782 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 10 client.4650.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 10 monclient: _renew_subs 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441e5940 con 0x7fc0440c99a0 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.766+0000 7fc04a858640 1 librados: init done 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.767+0000 7fc04a858640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.767+0000 7fc04a858640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.767+0000 7fc04a858640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:38.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.767+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044070300 con 0x7fc0440c99a0 2026-02-24T18:35:38.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.778+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc034003710 con 0x7fc0440c99a0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.778+0000 7fc040ff9640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 0x7fc01407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.778+0000 7fc043fff640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 0x7fc01407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.778+0000 7fc043fff640 10 monclient: get_auth_request con 0x7fc014078e30 auth_method 0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.779+0000 7fc043fff640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 0x7fc01407b2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc030005980 tx=0x7fc030007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.779+0000 7fc040ff9640 10 client.4650.objecter ms_handle_connect 0x7fc014078e30 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.780+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fc03402fa60 con 0x7fc0440c99a0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.780+0000 7fc040ff9640 10 client.4650.objecter ms_dispatch2 0x7fc044000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.780+0000 7fc040ff9640 3 client.4650.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.780+0000 7fc040ff9640 3 client.4650.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:38.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.780+0000 7fc040ff9640 20 client.4650.objecter dump_active .. 0 homeless 2026-02-24T18:35:38.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.878+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc034050070 con 0x7fc0440c99a0 2026-02-24T18:35:38.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.878+0000 7fc040ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:38.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.878+0000 7fc040ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:38.961 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.959+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0440c56c0 con 0x7fc014078e30 2026-02-24T18:35:38.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.961+0000 7fc040ff9640 1 -- 10.20.193.18:0/2017513827 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44852 (secure 0 0 0) 0x7fc0440c56c0 con 0x7fc014078e30 2026-02-24T18:35:38.963 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 10 librados: watch_flush enter 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 10 librados: watch_flush exit 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 20 client.4650.objecter shutdown clearing up homeless session... 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 10 client.4650.objecter successfully canceled tick 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 msgr2=0x7fc01407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 0x7fc01407b2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc030005980 tx=0x7fc030007dd0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 10 monclient: shutdown 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 msgr2=0x7fc0441e6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:38.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc03403a040 tx=0x7fc0340047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 shutdown_connections 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc014078e30 0x7fc01407b2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc04415a5e0 0x7fc0440cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc0440c99a0 0x7fc0441e6420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 --2- 10.20.193.18:0/2017513827 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc04406ff20 0x7fc04414c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 >> 10.20.193.18:0/2017513827 conn(0x7fc0440cd780 msgr2=0x7fc0440cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 shutdown_connections 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 -- 10.20.193.18:0/2017513827 wait complete. 2026-02-24T18:35:38.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:38.962+0000 7fc04a858640 1 librados: shutdown 2026-02-24T18:35:38.972 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2026-02-24T18:35:37.807819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328211,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":43,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":436816,"kb_used_data":5312,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512624,"statfs":{"total":1546188226560,"available":1545740926976,"internally_reserved":0,"allocated":5439488,"data_stored":2542916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":328211,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":43,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.497144"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.818354+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:35:32.818354+0000","last_peered":"2026-02-24T18:35:32.818354+0000","last_clean":"2026-02-24T18:35:32.818354+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:35:32.818354+0000","last_undegraded":"2026-02-24T18:35:32.818354+0000","last_fullsized":"2026-02-24T18:35:32.818354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.652906+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:35:32.652906+0000","last_peered":"2026-02-24T18:35:32.652906+0000","last_clean":"2026-02-24T18:35:32.652906+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:35:32.652906+0000","last_undegraded":"2026-02-24T18:35:32.652906+0000","last_fullsized":"2026-02-24T18:35:32.652906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.450105+0000","last_change":"2026-02-24T18:35:32.450105+0000","last_active":"2026-02-24T18:35:32.450105+0000","last_peered":"2026-02-24T18:35:32.450105+0000","last_clean":"2026-02-24T18:35:32.450105+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:35:32.450105+0000","last_undegraded":"2026-02-24T18:35:32.450105+0000","last_fullsized":"2026-02-24T18:35:32.450105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:08:13.272990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.901146+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:35:32.901146+0000","last_peered":"2026-02-24T18:35:32.901146+0000","last_clean":"2026-02-24T18:35:32.901146+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:35:32.901146+0000","last_undegraded":"2026-02-24T18:35:32.901146+0000","last_fullsized":"2026-02-24T18:35:32.901146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.520429+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:35:32.520429+0000","last_peered":"2026-02-24T18:35:32.520429+0000","last_clean":"2026-02-24T18:35:32.520429+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:35:32.520429+0000","last_undegraded":"2026-02-24T18:35:32.520429+0000","last_fullsized":"2026-02-24T18:35:32.520429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:33.317758+0000","last_change":"2026-02-24T18:35:33.340094+0000","last_active":"2026-02-24T18:35:33.317758+0000","last_peered":"2026-02-24T18:35:33.317758+0000","last_clean":"2026-02-24T18:35:33.317758+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:35:33.317758+0000","last_undegraded":"2026-02-24T18:35:33.317758+0000","last_fullsized":"2026-02-24T18:35:33.317758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:14.613111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.500422+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:35:32.500422+0000","last_peered":"2026-02-24T18:35:32.500422+0000","last_clean":"2026-02-24T18:35:32.500422+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:35:32.500422+0000","last_undegraded":"2026-02-24T18:35:32.500422+0000","last_fullsized":"2026-02-24T18:35:32.500422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.428660+0000","last_change":"2026-02-24T18:35:32.447249+0000","last_active":"2026-02-24T18:35:32.428660+0000","last_peered":"2026-02-24T18:35:32.428660+0000","last_clean":"2026-02-24T18:35:32.428660+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:35:32.428660+0000","last_undegraded":"2026-02-24T18:35:32.428660+0000","last_fullsized":"2026-02-24T18:35:32.428660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:27:55.602090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"25'19","reported_seq":46,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:33.295384+0000","last_change":"2026-02-24T18:35:28.845690+0000","last_active":"2026-02-24T18:35:33.295384+0000","last_peered":"2026-02-24T18:35:33.295384+0000","last_clean":"2026-02-24T18:35:33.295384+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:35:33.295384+0000","last_undegraded":"2026-02-24T18:35:33.295384+0000","last_fullsized":"2026-02-24T18:35:33.295384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_clean_scrub_stamp":"2026-02-24T18:35:26.937667+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:13.898984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":656384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":303104,"data_stored":118521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":282624,"data_stored":108964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":303104,"data_stored":118521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":630784,"data_stored":446694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247812,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":630784,"data_stored":446694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:35:38.973 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:35:39.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 10 monclient: get_monmap_and_config 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 10 monclient: build_initial_monmap 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 10 monclient: monmap: 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:39.044043+0000 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:39.044043+0000 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:39.045 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:39.046 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 1 Processor -- start 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 1 -- start start 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.043+0000 7feff3a88640 10 monclient: init 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient: _add_conns ranks=[4,3,2,0,5,1] 2026-02-24T18:35:39.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fefec1527f0 0x7fefec073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-b con 0x7fefec1527f0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fefec15d2c0 0x7fefec15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-a con 0x7fefec15d2c0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fefec15d730 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefec073c20 con 0x7fefec15d730 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fefec074b10 con 0x7fefec15d2c0 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fefec073f00 con 0x7fefec1527f0 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff3a88640 10 monclient(hunting): authenticate will time out at 1482.725338s 2026-02-24T18:35:39.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff0ffc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fefec15d2c0 0x7fefec15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff17fd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fefec1527f0 0x7fefec073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff1ffe640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fefec15d730 0x7fefec0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:58458/0 (socket says 10.20.193.18:58458) 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff1ffe640 1 -- 10.20.193.18:0/3788620517 learned_addr learned my addr 10.20.193.18:0/3788620517 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff17fd640 10 monclient(hunting): get_auth_request con 0x7fefec1527f0 auth_method 0 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff17fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff17fd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff17fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff0ffc640 10 monclient(hunting): get_auth_request con 0x7fefec15d2c0 auth_method 0 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff0ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff0ffc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.044+0000 7feff0ffc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.045+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 1 -- 10.20.193.18:0/3788620517 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201060337 0 0) 0x7fefec073c20 con 0x7fefec15d730 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 10 monclient(hunting): _negotiate 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 10 monclient(hunting): my global_id is 4601 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7fefdbfff640 1 -- 10.20.193.18:0/3788620517 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd4003620 con 0x7fefec15d730 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 10 monclient(hunting): handle_auth_done global_id 4615 payload 274 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 1 -- 10.20.193.18:0/3788620517 >> v2:10.20.193.18:3300/0 conn(0x7fefec15d2c0 msgr2=0x7fefec15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:39.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 1 --2- 10.20.193.18:0/3788620517 >> v2:10.20.193.18:3300/0 conn(0x7fefec15d2c0 0x7fefec15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 1 -- 10.20.193.18:0/3788620517 >> v1:10.20.193.110:6789/0 conn(0x7fefec15d730 legacy=0x7fefec0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 1 monclient: found mon.noname-b 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.046+0000 7feff17fd640 1 -- 10.20.193.18:0/3788620517 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec0741e0 con 0x7fefec1527f0 2026-02-24T18:35:39.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.047+0000 7feff17fd640 1 --2- 10.20.193.18:0/3788620517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec073530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefec06e9a0 tx=0x7fefdc02ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3f8580cbae0954 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 1 -- 10.20.193.18:0/3788620517 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc03c070 con 0x7fefec1527f0 2026-02-24T18:35:39.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 10 monclient: dump: 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.060 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 10 monclient: _finish_auth 0 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7fefdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7feff3a88640 5 monclient: authenticate success, global_id 4615 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.048+0000 7feff3a88640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 1 -- 10.20.193.18:0/3788620517 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc02faf0 con 0x7fefec1527f0 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 1 -- 10.20.193.18:0/3788620517 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc037440 con 0x7fefec1527f0 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 4 set_mon_vals no callback set 2026-02-24T18:35:39.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7fefdbfff640 10 monclient: dump: 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 10 monclient: shutdown 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 -- 10.20.193.18:0/3788620517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 msgr2=0x7fefec073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 --2- 10.20.193.18:0/3788620517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec073530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefec06e9a0 tx=0x7fefdc02ede0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 -- 10.20.193.18:0/3788620517 shutdown_connections 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 --2- 10.20.193.18:0/3788620517 >> v2:10.20.193.18:3300/0 conn(0x7fefec15d2c0 0x7fefec15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 --2- 10.20.193.18:0/3788620517 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec073530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 -- 10.20.193.18:0/3788620517 >> 10.20.193.18:0/3788620517 conn(0x7fefec0ce730 msgr2=0x7fefec0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 -- 10.20.193.18:0/3788620517 shutdown_connections 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 -- 10.20.193.18:0/3788620517 wait complete. 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 10 monclient: build_initial_monmap 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 10 monclient: monmap: 2026-02-24T18:35:39.063 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 librados: starting msgr at 2026-02-24T18:35:39.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.049+0000 7feff3a88640 1 librados: starting objecter 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 Processor -- start 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 -- start start 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 librados: setting wanted keys 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 librados: calling monclient init 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient: init 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-b con 0x7fefec1527f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fefec15d2c0 0x7fefec157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-c con 0x7fefec15d2c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 0x7fefec15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): picked mon.noname-a con 0x7fefec15d730 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fefec074fa0 con 0x7fefec15d730 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefec0747a0 con 0x7fefec1527f0 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fefec075120 con 0x7fefec15d2c0 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff3a88640 10 monclient(hunting): authenticate will time out at 1482.731506s 2026-02-24T18:35:39.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff17fd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff1ffe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 0x7fefec15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff0ffc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fefec15d2c0 0x7fefec157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff17fd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53976/0 (socket says 10.20.193.18:53976) 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff1ffe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 0x7fefec15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42200/0 (socket says 10.20.193.18:42200) 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff17fd640 1 -- 10.20.193.18:0/2384788707 learned_addr learned my addr 10.20.193.18:0/2384788707 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.050+0000 7feff17fd640 10 monclient(hunting): get_auth_request con 0x7fefec1527f0 auth_method 0 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): get_auth_request con 0x7fefec15d730 auth_method 0 2026-02-24T18:35:39.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): get_auth_request con 0x7fefec15d2c0 auth_method 0 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff17fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff0ffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.051+0000 7feff1ffe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 10 monclient(hunting): handle_auth_done global_id 4618 payload 824 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 -- 10.20.193.18:0/2384788707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fefec15d2c0 msgr2=0x7fefec157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fefec15d2c0 0x7fefec157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 -- 10.20.193.18:0/2384788707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 msgr2=0x7fefec15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 0x7fefec15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 monclient: found mon.noname-b 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.052+0000 7feff17fd640 1 -- 10.20.193.18:0/2384788707 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec1f3a10 con 0x7fefec1527f0 2026-02-24T18:35:39.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.053+0000 7feff17fd640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fefdc002410 tx=0x7fefdc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.053+0000 7fefd9ffb640 10 client.?.objecter ms_handle_connect 0x7fefec1527f0 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.053+0000 7fefd9ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc03c040 con 0x7fefec1527f0 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: dump: 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.070 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefdc0039d0 con 0x7fefec1527f0 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003d80 con 0x7fefec1527f0 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:39.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefd9ffb640 10 monclient: dump: 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7fefdb7fe640 4 set_mon_vals no callback set 2026-02-24T18:35:39.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 5 monclient: authenticate success, global_id 4618 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _renew_subs 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec1f3cf0 con 0x7fefec1527f0 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 client.4618.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _renew_subs 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1f42b0 con 0x7fefec1527f0 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 1 librados: init done 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:35:39.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.055+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec152bd0 con 0x7fefec1527f0 2026-02-24T18:35:39.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.161+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fefdc037440 con 0x7fefec1527f0 2026-02-24T18:35:39.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.162+0000 7fefd9ffb640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 0x7fefbc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.162+0000 7feff0ffc640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 0x7fefbc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.162+0000 7feff0ffc640 10 monclient: get_auth_request con 0x7fefbc078e00 auth_method 0 2026-02-24T18:35:39.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.163+0000 7fefd9ffb640 10 client.4618.objecter ms_handle_connect 0x7fefbc078e00 2026-02-24T18:35:39.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.163+0000 7feff0ffc640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 0x7fefbc07b2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fefec156410 tx=0x7fefe0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.165+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8972+0+0 (secure 0 0 0) 0x7fefdc083400 con 0x7fefec1527f0 2026-02-24T18:35:39.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.165+0000 7fefd9ffb640 10 client.4618.objecter ms_dispatch2 0x7fefec000b90 osd_map(28..28 src has 1..28) 2026-02-24T18:35:39.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.165+0000 7fefd9ffb640 3 client.4618.objecter handle_osd_map got epochs [28,28] > 0 2026-02-24T18:35:39.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.165+0000 7fefd9ffb640 3 client.4618.objecter handle_osd_map decoding full epoch 28 2026-02-24T18:35:39.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.166+0000 7fefd9ffb640 20 client.4618.objecter dump_active .. 0 homeless 2026-02-24T18:35:39.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.172+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fefdc0873c0 con 0x7fefec1527f0 2026-02-24T18:35:39.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.172+0000 7fefd9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.172+0000 7fefd9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:39.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.254+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefec073940 con 0x7fefbc078e00 2026-02-24T18:35:39.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.255+0000 7fefd9ffb640 1 -- 10.20.193.18:0/2384788707 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44860 (secure 0 0 0) 0x7fefec073940 con 0x7fefbc078e00 2026-02-24T18:35:39.257 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 10 librados: watch_flush enter 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 10 librados: watch_flush exit 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 20 client.4618.objecter shutdown clearing up homeless session... 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 10 client.4618.objecter successfully canceled tick 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 msgr2=0x7fefbc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 0x7fefbc07b2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fefec156410 tx=0x7fefe0009290 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 10 monclient: shutdown 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 msgr2=0x7fefec1578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fefdc002410 tx=0x7fefdc0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 shutdown_connections 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fefbc078e00 0x7fefbc07b2c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fefec15d730 0x7fefec15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fefec15d2c0 0x7fefec157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 --2- 10.20.193.18:0/2384788707 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fefec1527f0 0x7fefec1578d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 >> 10.20.193.18:0/2384788707 conn(0x7fefec0ce730 msgr2=0x7fefec0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 shutdown_connections 2026-02-24T18:35:39.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 -- 10.20.193.18:0/2384788707 wait complete. 2026-02-24T18:35:39.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.256+0000 7feff3a88640 1 librados: shutdown 2026-02-24T18:35:39.267 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-24T18:35:39.245474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437200,"kb_used_data":5464,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512240,"statfs":{"total":1546188226560,"available":1545740533760,"internally_reserved":0,"allocated":5595136,"data_stored":2691750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":459299,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.934800"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.818354+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:35:32.818354+0000","last_peered":"2026-02-24T18:35:32.818354+0000","last_clean":"2026-02-24T18:35:32.818354+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:35:32.818354+0000","last_undegraded":"2026-02-24T18:35:32.818354+0000","last_fullsized":"2026-02-24T18:35:32.818354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.652906+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:35:32.652906+0000","last_peered":"2026-02-24T18:35:32.652906+0000","last_clean":"2026-02-24T18:35:32.652906+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:35:32.652906+0000","last_undegraded":"2026-02-24T18:35:32.652906+0000","last_fullsized":"2026-02-24T18:35:32.652906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.450105+0000","last_change":"2026-02-24T18:35:32.450105+0000","last_active":"2026-02-24T18:35:32.450105+0000","last_peered":"2026-02-24T18:35:32.450105+0000","last_clean":"2026-02-24T18:35:32.450105+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:35:32.450105+0000","last_undegraded":"2026-02-24T18:35:32.450105+0000","last_fullsized":"2026-02-24T18:35:32.450105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:08:13.272990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.901146+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:35:32.901146+0000","last_peered":"2026-02-24T18:35:32.901146+0000","last_clean":"2026-02-24T18:35:32.901146+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:35:32.901146+0000","last_undegraded":"2026-02-24T18:35:32.901146+0000","last_fullsized":"2026-02-24T18:35:32.901146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.520429+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:35:32.520429+0000","last_peered":"2026-02-24T18:35:32.520429+0000","last_clean":"2026-02-24T18:35:32.520429+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:35:32.520429+0000","last_undegraded":"2026-02-24T18:35:32.520429+0000","last_fullsized":"2026-02-24T18:35:32.520429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:33.317758+0000","last_change":"2026-02-24T18:35:33.340094+0000","last_active":"2026-02-24T18:35:33.317758+0000","last_peered":"2026-02-24T18:35:33.317758+0000","last_clean":"2026-02-24T18:35:33.317758+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:35:33.317758+0000","last_undegraded":"2026-02-24T18:35:33.317758+0000","last_fullsized":"2026-02-24T18:35:33.317758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:14.613111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.500422+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:35:32.500422+0000","last_peered":"2026-02-24T18:35:32.500422+0000","last_clean":"2026-02-24T18:35:32.500422+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:35:32.500422+0000","last_undegraded":"2026-02-24T18:35:32.500422+0000","last_fullsized":"2026-02-24T18:35:32.500422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:32.428660+0000","last_change":"2026-02-24T18:35:32.447249+0000","last_active":"2026-02-24T18:35:32.428660+0000","last_peered":"2026-02-24T18:35:32.428660+0000","last_clean":"2026-02-24T18:35:32.428660+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:35:32.428660+0000","last_undegraded":"2026-02-24T18:35:32.428660+0000","last_fullsized":"2026-02-24T18:35:32.428660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:27:55.602090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"28'32","reported_seq":62,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-24T18:35:37.369075+0000","last_change":"2026-02-24T18:35:28.845690+0000","last_active":"2026-02-24T18:35:37.369075+0000","last_peered":"2026-02-24T18:35:37.369075+0000","last_clean":"2026-02-24T18:35:37.369075+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:35:37.369075+0000","last_undegraded":"2026-02-24T18:35:37.369075+0000","last_fullsized":"2026-02-24T18:35:37.369075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_clean_scrub_stamp":"2026-02-24T18:35:26.937667+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:13.898984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247812,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":303104,"data_stored":118521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247812,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":303104,"data_stored":118521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":700416,"data_stored":516342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":700416,"data_stored":516342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":299008,"data_stored":118502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:35:39.269 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T18:35:39.269 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-24T18:35:39.269 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-24T18:35:39.269 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:39.279 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:17.225 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.308 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:17.254 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.333 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:17.279 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.335 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:17.280 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.336 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:17.243 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.336 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:17.244 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.336 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:17.281 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 10 monclient: get_monmap_and_config 2026-02-24T18:35:39.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 10 monclient: build_initial_monmap 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 10 monclient: monmap: 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:39.338137+0000 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:39.338137+0000 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:39.339 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:39.340 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 1 Processor -- start 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 1 -- start start 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.337+0000 7f52bafeb640 10 monclient: init 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient: _add_conns ranks=[5,1,3,0,2,4] 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f52b4155dd0 0x7f52b4071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-c con 0x7f52b4155dd0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f52b415ffc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f52b41561b0 0x7f52b40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-a con 0x7f52b41561b0 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:39.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b4073960 con 0x7f52b415ffc0 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f52b4073710 con 0x7f52b41561b0 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f52b4073c40 con 0x7f52b4155dd0 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:39.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.338+0000 7f52bafeb640 10 monclient(hunting): authenticate will time out at 1483.019474s 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.340+0000 7f52b9561640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f52b41561b0 0x7f52b40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.340+0000 7f52b9561640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f52b41561b0 0x7f52b40c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46922/0 (socket says 10.20.193.18:46922) 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.341+0000 7f52b9561640 1 -- 10.20.193.18:0/1083639195 learned_addr learned my addr 10.20.193.18:0/1083639195 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.341+0000 7f52b9561640 10 monclient(hunting): get_auth_request con 0x7f52b41561b0 auth_method 0 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.341+0000 7f52b9561640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.341+0000 7f52b9561640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.341+0000 7f52b9561640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.342+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.342+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.342+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 10 monclient(hunting): handle_auth_done global_id 4659 payload 274 2026-02-24T18:35:39.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 1 -- 10.20.193.18:0/1083639195 >> v2:10.20.193.110:3300/0 conn(0x7f52b4155dd0 msgr2=0x7f52b4071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:39.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 1 --2- 10.20.193.18:0/1083639195 >> v2:10.20.193.110:3300/0 conn(0x7f52b4155dd0 0x7f52b4071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 1 -- 10.20.193.18:0/1083639195 >> v1:10.20.193.32:6789/0 conn(0x7f52b415ffc0 legacy=0x7f52b4076250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:39.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 10 monclient: _finish_hunting 0 2026-02-24T18:35:39.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 1 monclient: found mon.noname-a 2026-02-24T18:35:39.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:39.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:39.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.344+0000 7f52b9561640 1 -- 10.20.193.18:0/1083639195 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b4073f20 con 0x7f52b41561b0 2026-02-24T18:35:39.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.346+0000 7f52b9561640 1 --2- 10.20.193.18:0/1083639195 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b41561b0 0x7f52b40c9020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52b4072850 tx=0x7f52b002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ba8d7ec092308 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 1 -- 10.20.193.18:0/1083639195 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002fc50 con 0x7f52b41561b0 2026-02-24T18:35:39.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: dump: 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.355 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 5 monclient: authenticate success, global_id 4659 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 1 -- 10.20.193.18:0/1083639195 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b002fdf0 con 0x7f52b41561b0 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 1 -- 10.20.193.18:0/1083639195 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0037610 con 0x7f52b41561b0 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 4 set_mon_vals no callback set 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:39.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52ab7fe640 10 monclient: dump: 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 10 monclient: shutdown 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:39.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 -- 10.20.193.18:0/1083639195 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b41561b0 msgr2=0x7f52b40c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 --2- 10.20.193.18:0/1083639195 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b41561b0 0x7f52b40c9020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52b4072850 tx=0x7f52b002f230 comp rx=0 tx=0).stop 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 -- 10.20.193.18:0/1083639195 shutdown_connections 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 --2- 10.20.193.18:0/1083639195 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b41561b0 0x7f52b40c9020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 --2- 10.20.193.18:0/1083639195 >> v2:10.20.193.110:3300/0 conn(0x7f52b4155dd0 0x7f52b4071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 -- 10.20.193.18:0/1083639195 >> 10.20.193.18:0/1083639195 conn(0x7f52b40ca710 msgr2=0x7f52b40cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 -- 10.20.193.18:0/1083639195 shutdown_connections 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.351+0000 7f52bafeb640 1 -- 10.20.193.18:0/1083639195 wait complete. 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient: build_initial_monmap 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient: monmap: 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:39.358 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 librados: starting msgr at 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 librados: starting objecter 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 Processor -- start 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 -- start start 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 librados: setting wanted keys 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 librados: calling monclient init 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient: init 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:39.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b4155dd0 0x7f52b41f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-a con 0x7f52b4155dd0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f52b41561b0 0x7f52b41f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-b con 0x7f52b41561b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): picked mon.noname-c con 0x7f52b415ffc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f52b40c9560 con 0x7f52b4155dd0 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52b40c8b00 con 0x7f52b41561b0 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52b40c96e0 con 0x7f52b415ffc0 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:39.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.352+0000 7f52bafeb640 10 monclient(hunting): authenticate will time out at 1483.033800s 2026-02-24T18:35:39.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.353+0000 7f52b9561640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.353+0000 7f52abfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f52b41561b0 0x7f52b41f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:34706/0 (socket says 10.20.193.18:34706) 2026-02-24T18:35:39.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 1 -- 10.20.193.18:0/602784789 learned_addr learned my addr 10.20.193.18:0/602784789 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): get_auth_request con 0x7f52b415ffc0 auth_method 0 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52abfff640 10 monclient(hunting): get_auth_request con 0x7f52b41561b0 auth_method 0 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52abfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.354+0000 7f52b9561640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.355+0000 7f52abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.355+0000 7f52abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.355+0000 7f52abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.356+0000 7f52b8d60640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b4155dd0 0x7f52b41f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 10 monclient(hunting): handle_auth_done global_id 4607 payload 824 2026-02-24T18:35:39.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 -- 10.20.193.18:0/602784789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f52b41561b0 msgr2=0x7f52b41f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f52b41561b0 0x7f52b41f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 -- 10.20.193.18:0/602784789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b4155dd0 msgr2=0x7f52b41f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b4155dd0 0x7f52b41f7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 10 monclient: _finish_hunting 0 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 monclient: found mon.noname-c 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.357+0000 7f52b9561640 1 -- 10.20.193.18:0/602784789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41f4990 con 0x7f52b415ffc0 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.359+0000 7f52b9561640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52b0039040 tx=0x7f52b0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.359+0000 7f52a97fa640 10 client.?.objecter ms_handle_connect 0x7f52b415ffc0 2026-02-24T18:35:39.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.359+0000 7f52a97fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:39.365 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:17.309 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0046070 con 0x7f52b415ffc0 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: dump: 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.366 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0042640 con 0x7f52b415ffc0 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0042a10 con 0x7f52b415ffc0 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52aaffd640 4 set_mon_vals no callback set 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52a97fa640 10 monclient: dump: 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:39.367 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52bafeb640 5 monclient: authenticate success, global_id 4607 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52bafeb640 10 monclient: _renew_subs 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52bafeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.362+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b40bfd50 con 0x7f52b415ffc0 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 client.4607.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 monclient: _renew_subs 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b40c0310 con 0x7f52b415ffc0 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 1 librados: init done 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:39.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.363+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b4074ec0 con 0x7f52b415ffc0 2026-02-24T18:35:39.369 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:17.273 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:39.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.471+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f52b0042bb0 con 0x7f52b415ffc0 2026-02-24T18:35:39.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.472+0000 7f52a97fa640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 0x7f528407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:39.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.472+0000 7f52b8d60640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 0x7f528407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:39.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.472+0000 7f52b8d60640 10 monclient: get_auth_request con 0x7f5284078e50 auth_method 0 2026-02-24T18:35:39.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.472+0000 7f52a97fa640 10 client.4607.objecter ms_handle_connect 0x7f5284078e50 2026-02-24T18:35:39.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.472+0000 7f52b8d60640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 0x7f528407b310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52a400a8b0 tx=0x7f52a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:39.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.474+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f52b0083f00 con 0x7f52b415ffc0 2026-02-24T18:35:39.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.474+0000 7f52a97fa640 10 client.4607.objecter ms_dispatch2 0x7f52b4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:39.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.474+0000 7f52a97fa640 3 client.4607.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:39.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.474+0000 7f52a97fa640 3 client.4607.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:39.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.474+0000 7f52a97fa640 20 client.4607.objecter dump_active .. 0 homeless 2026-02-24T18:35:39.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.480+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f52b4074ec0 con 0x7f52b415ffc0 2026-02-24T18:35:39.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.480+0000 7f52a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:39.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.480+0000 7f52a97fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:39.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.582+0000 7f52bafeb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:39.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.582+0000 7f52bafeb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:39.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.582+0000 7f52bafeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:39.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.582+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f52b4071360 con 0x7f52b415ffc0 2026-02-24T18:35:39.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.588+0000 7f52a97fa640 1 -- 10.20.193.18:0/602784789 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f52b00885c0 con 0x7f52b415ffc0 2026-02-24T18:35:39.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.588+0000 7f52a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:39.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.588+0000 7f52a97fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:39.590 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:39.590 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:39.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 10 librados: watch_flush enter 2026-02-24T18:35:39.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 10 librados: watch_flush exit 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 20 client.4607.objecter shutdown clearing up homeless session... 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 10 client.4607.objecter successfully canceled tick 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 msgr2=0x7f528407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 0x7f528407b310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52a400a8b0 tx=0x7f52a4008040 comp rx=0 tx=0).stop 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 10 monclient: shutdown 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 msgr2=0x7f52b414ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52b0039040 tx=0x7f52b0004b40 comp rx=0 tx=0).stop 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 shutdown_connections 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5284078e50 0x7f528407b310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f52b41561b0 0x7f52b41f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f52b415ffc0 0x7f52b414ed40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 --2- 10.20.193.18:0/602784789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f52b4155dd0 0x7f52b41f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 >> 10.20.193.18:0/602784789 conn(0x7f52b40ca710 msgr2=0x7f52b4150070 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 shutdown_connections 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 -- 10.20.193.18:0/602784789 wait complete. 2026-02-24T18:35:39.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:39.589+0000 7f52bafeb640 1 librados: shutdown 2026-02-24T18:35:42.600 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:42.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 10 monclient: get_monmap_and_config 2026-02-24T18:35:42.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 10 monclient: build_initial_monmap 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 10 monclient: monmap: 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:42.670307+0000 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:42.670307+0000 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 1 Processor -- start 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 1 -- start start 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.668+0000 7efec952c640 10 monclient: init 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient: _add_conns ranks=[3,1,0,4,2,5] 2026-02-24T18:35:42.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efec406ff20 0x7efec4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): picked mon.noname-a con 0x7efec406ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efec40c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efec415a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec40c7e90 con 0x7efec415a5e0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec40c8170 con 0x7efec40c99a0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7efec4159a40 con 0x7efec406ff20 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec952c640 10 monclient(hunting): authenticate will time out at 1486.351457s 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec2ffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efec406ff20 0x7efec4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec27fc640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7efec40c99a0 0x7efec4159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:38148/0 (socket says 10.20.193.18:38148) 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.669+0000 7efec27fc640 1 -- 10.20.193.18:0/1994267358 learned_addr learned my addr 10.20.193.18:0/1994267358 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): get_auth_request con 0x7efec406ff20 auth_method 0 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec37fe640 1 -- 10.20.193.18:0/1994267358 >> v1:10.20.193.18:6789/0 conn(0x7efec415a5e0 legacy=0x7efec414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec37fe640 1 -- 10.20.193.18:0/1994267358 >> v1:10.20.193.18:6789/0 conn(0x7efec415a5e0 legacy=0x7efec414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec37fe640 1 --1- 10.20.193.18:0/1994267358 >> v1:10.20.193.18:6789/0 conn(0x7efec415a5e0 0x7efec414fe10 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:42.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.670+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 1 -- 10.20.193.18:0/1994267358 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820655938 0 0) 0x7efec40c8170 con 0x7efec40c99a0 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 10 monclient(hunting): my global_id is 4624 2026-02-24T18:35:42.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.671+0000 7efec1ffb640 1 -- 10.20.193.18:0/1994267358 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb0003620 con 0x7efec40c99a0 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 10 monclient(hunting): handle_auth_done global_id 4662 payload 274 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 1 -- 10.20.193.18:0/1994267358 >> v1:10.20.193.32:6789/0 conn(0x7efec40c99a0 legacy=0x7efec4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 1 -- 10.20.193.18:0/1994267358 >> v1:10.20.193.18:6789/0 conn(0x7efec415a5e0 legacy=0x7efec414fe10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 1 monclient: found mon.noname-a 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.672+0000 7efec2ffd640 1 -- 10.20.193.18:0/1994267358 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec40c8450 con 0x7efec406ff20 2026-02-24T18:35:42.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.673+0000 7efec2ffd640 1 --2- 10.20.193.18:0/1994267358 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efeb80098d0 tx=0x7efeb802ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8de3c590929aa4 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 1 -- 10.20.193.18:0/1994267358 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb803c070 con 0x7efec406ff20 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: dump: 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:42.677 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 1 -- 10.20.193.18:0/1994267358 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb802fb00 con 0x7efec406ff20 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 5 monclient: authenticate success, global_id 4662 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 1 -- 10.20.193.18:0/1994267358 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8037440 con 0x7efec406ff20 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 4 set_mon_vals no callback set 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec1ffb640 10 monclient: dump: 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 10 monclient: shutdown 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 -- 10.20.193.18:0/1994267358 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 msgr2=0x7efec4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 --2- 10.20.193.18:0/1994267358 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efeb80098d0 tx=0x7efeb802ee80 comp rx=0 tx=0).stop 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 -- 10.20.193.18:0/1994267358 shutdown_connections 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 --2- 10.20.193.18:0/1994267358 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4070300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 -- 10.20.193.18:0/1994267358 >> 10.20.193.18:0/1994267358 conn(0x7efec40cd780 msgr2=0x7efec40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 -- 10.20.193.18:0/1994267358 shutdown_connections 2026-02-24T18:35:42.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 1 -- 10.20.193.18:0/1994267358 wait complete. 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 10 monclient: build_initial_monmap 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.675+0000 7efec952c640 10 monclient: monmap: 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 librados: starting msgr at 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 librados: starting objecter 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 Processor -- start 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 -- start start 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 librados: setting wanted keys 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 librados: calling monclient init 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 10 monclient: init 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:42.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 10 monclient(hunting): picked mon.noname-a con 0x7efec406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.676+0000 7efec952c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efec40c99a0 0x7efec40c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): picked mon.noname-b con 0x7efec40c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efec415a5e0 0x7efec40c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): picked mon.noname-c con 0x7efec415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efec4150520 con 0x7efec406ff20 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efec40c8a10 con 0x7efec40c99a0 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efec41506a0 con 0x7efec415a5e0 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec952c640 10 monclient(hunting): authenticate will time out at 1486.359031s 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46966/0 (socket says 10.20.193.18:46966) 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 1 -- 10.20.193.18:0/2733680402 learned_addr learned my addr 10.20.193.18:0/2733680402 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): get_auth_request con 0x7efec406ff20 auth_method 0 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:42.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec27fc640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efec40c99a0 0x7efec40c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec27fc640 10 monclient(hunting): get_auth_request con 0x7efec40c99a0 auth_method 0 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec37fe640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efec415a5e0 0x7efec40c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec37fe640 10 monclient(hunting): get_auth_request con 0x7efec415a5e0 auth_method 0 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.677+0000 7efec37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.678+0000 7efec37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 10 monclient(hunting): handle_auth_done global_id 4665 payload 824 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 -- 10.20.193.18:0/2733680402 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efec415a5e0 msgr2=0x7efec40c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efec415a5e0 0x7efec40c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 -- 10.20.193.18:0/2733680402 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efec40c99a0 msgr2=0x7efec40c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efec40c99a0 0x7efec40c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 monclient: found mon.noname-a 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:42.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.679+0000 7efec2ffd640 1 -- 10.20.193.18:0/2733680402 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec40c1170 con 0x7efec406ff20 2026-02-24T18:35:42.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.680+0000 7efec2ffd640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efeb803a040 tx=0x7efeb80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:42.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.680+0000 7efea37fe640 10 client.?.objecter ms_handle_connect 0x7efec406ff20 2026-02-24T18:35:42.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.680+0000 7efea37fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8046020 con 0x7efec406ff20 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: dump: 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb8042620 con 0x7efec406ff20 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb803c040 con 0x7efec406ff20 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efea37fe640 10 monclient: dump: 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.684 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 5 monclient: authenticate success, global_id 4665 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec17fa640 4 set_mon_vals no callback set 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 10 monclient: _renew_subs 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41f1780 con 0x7efec406ff20 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 10 client.4665.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 10 monclient: _renew_subs 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41f1d20 con 0x7efec406ff20 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.681+0000 7efec952c640 1 librados: init done 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.682+0000 7efec952c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.682+0000 7efec952c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.682+0000 7efec952c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.682+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4076e00 con 0x7efec406ff20 2026-02-24T18:35:42.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.694+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efeb803f030 con 0x7efec406ff20 2026-02-24T18:35:42.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.695+0000 7efea37fe640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 0x7efe9007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:42.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.695+0000 7efec27fc640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 0x7efe9007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:42.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.695+0000 7efec27fc640 10 monclient: get_auth_request con 0x7efe90078e50 auth_method 0 2026-02-24T18:35:42.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 10 client.4665.objecter ms_handle_connect 0x7efe90078e50 2026-02-24T18:35:42.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efec27fc640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 0x7efe9007b310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efeac007b60 tx=0x7efeac007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:42.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7efeb802f9c0 con 0x7efec406ff20 2026-02-24T18:35:42.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 10 client.4665.objecter ms_dispatch2 0x7efec4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:42.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 3 client.4665.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:42.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 3 client.4665.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:42.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.696+0000 7efea37fe640 20 client.4665.objecter dump_active .. 0 homeless 2026-02-24T18:35:42.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.798+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efec4076e00 con 0x7efec406ff20 2026-02-24T18:35:42.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.798+0000 7efea37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:42.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.798+0000 7efea37fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:42.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.898+0000 7efec952c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:42.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.898+0000 7efec952c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:42.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.898+0000 7efec952c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:42.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.898+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efec4070300 con 0x7efec406ff20 2026-02-24T18:35:42.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.902+0000 7efea37fe640 1 -- 10.20.193.18:0/2733680402 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7efeb8050030 con 0x7efec406ff20 2026-02-24T18:35:42.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.902+0000 7efea37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:42.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.902+0000 7efea37fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:42.905 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:42.905 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 10 librados: watch_flush enter 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 10 librados: watch_flush exit 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 20 client.4665.objecter shutdown clearing up homeless session... 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 10 client.4665.objecter successfully canceled tick 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 msgr2=0x7efe9007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 0x7efe9007b310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efeac007b60 tx=0x7efeac007ab0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 10 monclient: shutdown 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 msgr2=0x7efec4072860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efeb803a040 tx=0x7efeb80047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 shutdown_connections 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe90078e50 0x7efe9007b310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efec40c99a0 0x7efec40c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efec415a5e0 0x7efec40c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 --2- 10.20.193.18:0/2733680402 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efec406ff20 0x7efec4072860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 >> 10.20.193.18:0/2733680402 conn(0x7efec40cd780 msgr2=0x7efec414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 shutdown_connections 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 -- 10.20.193.18:0/2733680402 wait complete. 2026-02-24T18:35:42.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:42.903+0000 7efec952c640 1 librados: shutdown 2026-02-24T18:35:45.916 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 10 monclient: get_monmap_and_config 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 10 monclient: monmap: 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:45.984046+0000 2026-02-24T18:35:45.985 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:45.984046+0000 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 1 Processor -- start 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 1 -- start start 2026-02-24T18:35:45.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.983+0000 7f39869a9640 10 monclient: init 2026-02-24T18:35:45.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:45.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient: _add_conns ranks=[3,1,5,0,4,2] 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f398006ff20 0x7f3980070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-a con 0x7f398006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f39800c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f398015a5e0 0x7f398014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-c con 0x7f398015a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39800c7e90 con 0x7f39800c99a0 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3980159a40 con 0x7f398006ff20 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f39800c8170 con 0x7f398015a5e0 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:45.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f39869a9640 10 monclient(hunting): authenticate will time out at 1489.665180s 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f398006ff20 0x7f3980070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f3984f1f640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f398015a5e0 0x7f398014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f398006ff20 0x7f3980070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46980/0 (socket says 10.20.193.18:46980) 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 1 -- 10.20.193.18:0/1179960261 learned_addr learned my addr 10.20.193.18:0/1179960261 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 10 monclient(hunting): get_auth_request con 0x7f398006ff20 auth_method 0 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f397ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f3984f1f640 10 monclient(hunting): get_auth_request con 0x7f398015a5e0 auth_method 0 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f3984f1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f3984f1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:45.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.984+0000 7f3984f1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 1 -- 10.20.193.18:0/1179960261 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687268227 0 0) 0x7f39800c7e90 con 0x7f39800c99a0 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 10 monclient(hunting): _negotiate 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 10 monclient(hunting): my global_id is 4630 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397effd640 1 -- 10.20.193.18:0/1179960261 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3964003620 con 0x7f39800c99a0 2026-02-24T18:35:45.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient(hunting): handle_auth_done global_id 4668 payload 274 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 1 -- 10.20.193.18:0/1179960261 >> v2:10.20.193.110:3300/0 conn(0x7f398015a5e0 msgr2=0x7f398014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 1 --2- 10.20.193.18:0/1179960261 >> v2:10.20.193.110:3300/0 conn(0x7f398015a5e0 0x7f398014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 1 -- 10.20.193.18:0/1179960261 >> v1:10.20.193.32:6789/0 conn(0x7f39800c99a0 legacy=0x7f3980159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 1 monclient: found mon.noname-a 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.985+0000 7f397ffff640 1 -- 10.20.193.18:0/1179960261 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800c8450 con 0x7f398006ff20 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.986+0000 7f397ffff640 1 --2- 10.20.193.18:0/1179960261 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f3980070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f396c009970 tx=0x7f396c02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9a9d867750d134 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 1 -- 10.20.193.18:0/1179960261 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c03c070 con 0x7f398006ff20 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:45.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 10 monclient: dump: 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:45.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 10 monclient: _finish_auth 0 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 10 monclient: _check_auth_tickets 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f397effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f39869a9640 5 monclient: authenticate success, global_id 4668 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.987+0000 7f39869a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 1 -- 10.20.193.18:0/1179960261 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c02fbc0 con 0x7f398006ff20 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 1 -- 10.20.193.18:0/1179960261 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037440 con 0x7f398006ff20 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 4 set_mon_vals no callback set 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:46.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f397effd640 10 monclient: dump: 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 10 monclient: shutdown 2026-02-24T18:35:46.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 1 -- 10.20.193.18:0/1179960261 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 msgr2=0x7f3980070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.988+0000 7f39869a9640 1 --2- 10.20.193.18:0/1179960261 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f3980070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f396c009970 tx=0x7f396c02eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 -- 10.20.193.18:0/1179960261 shutdown_connections 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 --2- 10.20.193.18:0/1179960261 >> v2:10.20.193.110:3300/0 conn(0x7f398015a5e0 0x7f398014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 --2- 10.20.193.18:0/1179960261 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f3980070300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 -- 10.20.193.18:0/1179960261 >> 10.20.193.18:0/1179960261 conn(0x7f39800cd780 msgr2=0x7f39800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 -- 10.20.193.18:0/1179960261 shutdown_connections 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 -- 10.20.193.18:0/1179960261 wait complete. 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 10 monclient: build_initial_monmap 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 10 monclient: monmap: 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:46.002 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 librados: starting msgr at 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 librados: starting objecter 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 Processor -- start 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 -- start start 2026-02-24T18:35:46.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 librados: setting wanted keys 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 1 librados: calling monclient init 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.989+0000 7f39869a9640 10 monclient: init 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-a con 0x7f398006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f39800c99a0 0x7f3980148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-c con 0x7f39800c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398015a5e0 0x7f3980157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): picked mon.noname-b con 0x7f398015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3980150350 con 0x7f398006ff20 2026-02-24T18:35:46.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39800c8a10 con 0x7f398015a5e0 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39801504d0 con 0x7f39800c99a0 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f39869a9640 10 monclient(hunting): authenticate will time out at 1489.671122s 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46994/0 (socket says 10.20.193.18:46994) 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 1 -- 10.20.193.18:0/1313757463 learned_addr learned my addr 10.20.193.18:0/1313757463 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): get_auth_request con 0x7f398006ff20 auth_method 0 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:46.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397f7fe640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f39800c99a0 0x7f3980148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f3984f1f640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398015a5e0 0x7f3980157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397f7fe640 10 monclient(hunting): get_auth_request con 0x7f39800c99a0 auth_method 0 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f397f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f3984f1f640 10 monclient(hunting): get_auth_request con 0x7f398015a5e0 auth_method 0 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f3984f1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f3984f1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:46.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.990+0000 7f3984f1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f397f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f397f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f397f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.991+0000 7f3984f1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 10 monclient(hunting): handle_auth_done global_id 4671 payload 824 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 -- 10.20.193.18:0/1313757463 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f39800c99a0 msgr2=0x7f3980148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f39800c99a0 0x7f3980148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 -- 10.20.193.18:0/1313757463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398015a5e0 msgr2=0x7f3980157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398015a5e0 0x7f3980157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:46.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 monclient: found mon.noname-a 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 -- 10.20.193.18:0/1313757463 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3980158330 con 0x7f398006ff20 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397ffff640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f396c02f420 tx=0x7f396c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397cff9640 10 client.?.objecter ms_handle_connect 0x7f398006ff20 2026-02-24T18:35:46.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.992+0000 7f397cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:46.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c03c040 con 0x7f398006ff20 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: dump: 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:46.011 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: _finish_auth 0 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 5 monclient: authenticate success, global_id 4671 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c003b40 con 0x7f398006ff20 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 10 monclient: _renew_subs 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3980158610 con 0x7f398006ff20 2026-02-24T18:35:46.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037440 con 0x7f398006ff20 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397cff9640 10 monclient: dump: 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:46.013 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f397e7fc640 4 set_mon_vals no callback set 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 10 client.4671.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 10 monclient: _renew_subs 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39801547b0 con 0x7f398006ff20 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.994+0000 7f39869a9640 1 librados: init done 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.995+0000 7f39869a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.995+0000 7f39869a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.995+0000 7f39869a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:45.995+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39801542c0 con 0x7f398006ff20 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.006+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f396c0375e0 con 0x7f398006ff20 2026-02-24T18:35:46.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.007+0000 7f397cff9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 0x7f395007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:46.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.007+0000 7f397f7fe640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 0x7f395007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:46.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.007+0000 7f397f7fe640 10 monclient: get_auth_request con 0x7f3950078e50 auth_method 0 2026-02-24T18:35:46.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.007+0000 7f397f7fe640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 0x7f395007b310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3970002960 tx=0x7f39700028f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:46.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.007+0000 7f397cff9640 10 client.4671.objecter ms_handle_connect 0x7f3950078e50 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f396c083020 con 0x7f398006ff20 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 10 client.4671.objecter ms_dispatch2 0x7f3980000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 3 client.4671.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 3 client.4671.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 20 client.4671.objecter dump_active .. 0 homeless 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f396c086fe0 con 0x7f398006ff20 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:46.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.109+0000 7f397cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:46.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.211+0000 7f39869a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:46.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.211+0000 7f39869a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:46.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.211+0000 7f39869a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:46.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.211+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3980150860 con 0x7f398006ff20 2026-02-24T18:35:46.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.214+0000 7f397cff9640 1 -- 10.20.193.18:0/1313757463 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f396c08b2e0 con 0x7f398006ff20 2026-02-24T18:35:46.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.214+0000 7f397cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:46.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.214+0000 7f397cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:46.216 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:46.216 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 10 librados: watch_flush enter 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 10 librados: watch_flush exit 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 20 client.4671.objecter shutdown clearing up homeless session... 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 10 client.4671.objecter successfully canceled tick 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 msgr2=0x7f395007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 0x7f395007b310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3970002960 tx=0x7f39700028f0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 10 monclient: shutdown 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 msgr2=0x7f39801f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f396c02f420 tx=0x7f396c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 shutdown_connections 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3950078e50 0x7f395007b310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398015a5e0 0x7f3980157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f39800c99a0 0x7f3980148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 --2- 10.20.193.18:0/1313757463 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398006ff20 0x7f39801f41b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 >> 10.20.193.18:0/1313757463 conn(0x7f39800cd780 msgr2=0x7f398014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 shutdown_connections 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 -- 10.20.193.18:0/1313757463 wait complete. 2026-02-24T18:35:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:46.215+0000 7f39869a9640 1 librados: shutdown 2026-02-24T18:35:49.227 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:49.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.295+0000 7fbfbdbce640 10 monclient: get_monmap_and_config 2026-02-24T18:35:49.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.295+0000 7fbfbdbce640 10 monclient: build_initial_monmap 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient: monmap: 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:49.296862+0000 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:49.296862+0000 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:49.298 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:49.299 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:49.299 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 Processor -- start 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 -- start start 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient: init 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient: _add_conns ranks=[5,0,4,1,2,3] 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbfb806ff20 0x7fbfb8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-c con 0x7fbfb806ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbfb80c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:35:49.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fbfb815a5e0 0x7fbfb814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-b con 0x7fbfb815a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfb80c7e90 con 0x7fbfb80c99a0 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb8159a40 con 0x7fbfb815a5e0 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb80c8170 con 0x7fbfb806ff20 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.296+0000 7fbfbdbce640 10 monclient(hunting): authenticate will time out at 1492.977990s 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb6ffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fbfb80c99a0 0x7fbfb8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:43446/0 (socket says 10.20.193.18:43446) 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb6ffd640 1 -- 10.20.193.18:0/3391239419 learned_addr learned my addr 10.20.193.18:0/3391239419 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3391239419 >> v2:10.20.193.110:3300/0 conn(0x7fbfb806ff20 0x7fbfb8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb7fff640 1 --2- 10.20.193.18:0/3391239419 >> v2:10.20.193.32:3300/0 conn(0x7fbfb815a5e0 0x7fbfb814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb77fe640 10 monclient(hunting): get_auth_request con 0x7fbfb806ff20 auth_method 0 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb7fff640 10 monclient(hunting): get_auth_request con 0x7fbfb815a5e0 auth_method 0 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.297+0000 7fbfb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 1 -- 10.20.193.18:0/3391239419 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916348400 0 0) 0x7fbfb80c7e90 con 0x7fbfb80c99a0 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 10 monclient(hunting): _negotiate 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 10 monclient(hunting): my global_id is 4674 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.298+0000 7fbfb67fc640 1 -- 10.20.193.18:0/3391239419 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf9c003620 con 0x7fbfb80c99a0 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_done global_id 4619 payload 274 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3391239419 >> v2:10.20.193.32:3300/0 conn(0x7fbfb815a5e0 msgr2=0x7fbfb814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3391239419 >> v2:10.20.193.32:3300/0 conn(0x7fbfb815a5e0 0x7fbfb814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3391239419 >> v1:10.20.193.18:6789/0 conn(0x7fbfb80c99a0 legacy=0x7fbfb8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:49.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 1 monclient: found mon.noname-c 2026-02-24T18:35:49.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:49.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:49.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.299+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3391239419 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb80c8450 con 0x7fbfb806ff20 2026-02-24T18:35:49.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.300+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3391239419 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb806ff20 0x7fbfb8070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098a0 tx=0x7fbfa402ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65926d3d4a7841c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:49.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 1 -- 10.20.193.18:0/3391239419 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403c070 con 0x7fbfb806ff20 2026-02-24T18:35:49.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:49.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:49.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: dump: 2026-02-24T18:35:49.312 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:49.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 1 -- 10.20.193.18:0/3391239419 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa402fae0 con 0x7fbfb806ff20 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfbdbce640 5 monclient: authenticate success, global_id 4619 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 1 -- 10.20.193.18:0/3391239419 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4037440 con 0x7fbfb806ff20 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfb67fc640 10 monclient: dump: 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.314 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfbdbce640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.301+0000 7fbfbdbce640 4 set_mon_vals no callback set 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 10 monclient: shutdown 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3391239419 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb806ff20 msgr2=0x7fbfb8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3391239419 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb806ff20 0x7fbfb8070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098a0 tx=0x7fbfa402ee10 comp rx=0 tx=0).stop 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3391239419 shutdown_connections 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3391239419 >> v2:10.20.193.32:3300/0 conn(0x7fbfb815a5e0 0x7fbfb814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3391239419 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb806ff20 0x7fbfb8070300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3391239419 >> 10.20.193.18:0/3391239419 conn(0x7fbfb80cd780 msgr2=0x7fbfb80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3391239419 shutdown_connections 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3391239419 wait complete. 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 10 monclient: build_initial_monmap 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.302+0000 7fbfbdbce640 10 monclient: monmap: 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:49.316 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 librados: starting msgr at 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 librados: starting objecter 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 Processor -- start 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 -- start start 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 librados: setting wanted keys 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 librados: calling monclient init 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient: init 2026-02-24T18:35:49.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-a con 0x7fbfb806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbfb80c99a0 0x7fbfb806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-b con 0x7fbfb80c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb815a5e0 0x7fbfb81f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): picked mon.noname-c con 0x7fbfb815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8150350 con 0x7fbfb806ff20 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb80c8a10 con 0x7fbfb80c99a0 2026-02-24T18:35:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81504d0 con 0x7fbfb815a5e0 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfbdbce640 10 monclient(hunting): authenticate will time out at 1492.984697s 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.303+0000 7fbfb77fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49686/0 (socket says 10.20.193.18:49686) 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3317336170 learned_addr learned my addr 10.20.193.18:0/3317336170 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): get_auth_request con 0x7fbfb806ff20 auth_method 0 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbfb80c99a0 0x7fbfb806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb815a5e0 0x7fbfb81f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): get_auth_request con 0x7fbfb80c99a0 auth_method 0 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): get_auth_request con 0x7fbfb815a5e0 auth_method 0 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.304+0000 7fbfb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 10 monclient(hunting): handle_auth_done global_id 4677 payload 824 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3317336170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb815a5e0 msgr2=0x7fbfb81f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb815a5e0 0x7fbfb81f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3317336170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbfb80c99a0 msgr2=0x7fbfb806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbfb80c99a0 0x7fbfb806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 monclient: found mon.noname-a 2026-02-24T18:35:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.305+0000 7fbfb77fe640 1 -- 10.20.193.18:0/3317336170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb81f7ab0 con 0x7fbfb806ff20 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.306+0000 7fbfb77fe640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbfa403a040 tx=0x7fbfa40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.306+0000 7fbf93fff640 10 client.?.objecter ms_handle_connect 0x7fbfb806ff20 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.306+0000 7fbf93fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4046020 con 0x7fbfb806ff20 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: dump: 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:49.322 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: _finish_auth 0 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: _check_auth_tickets 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa4042430 con 0x7fbfb806ff20 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403c040 con 0x7fbfb806ff20 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbf93fff640 10 monclient: dump: 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:49.324 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfb5ffb640 4 set_mon_vals no callback set 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 5 monclient: authenticate success, global_id 4677 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 10 monclient: _renew_subs 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb81f41d0 con 0x7fbfb806ff20 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 10 client.4677.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 10 monclient: _renew_subs 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81f4b30 con 0x7fbfb806ff20 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.308+0000 7fbfbdbce640 1 librados: init done 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.309+0000 7fbfbdbce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.309+0000 7fbfbdbce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.309+0000 7fbfbdbce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.309+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb8150860 con 0x7fbfb806ff20 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.320+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbfa4053070 con 0x7fbfb806ff20 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.320+0000 7fbf93fff640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 0x7fbf8807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.321+0000 7fbfb6ffd640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 0x7fbf8807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.321+0000 7fbfb6ffd640 10 monclient: get_auth_request con 0x7fbf88078e00 auth_method 0 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.321+0000 7fbf93fff640 10 client.4677.objecter ms_handle_connect 0x7fbf88078e00 2026-02-24T18:35:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.321+0000 7fbfb6ffd640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 0x7fbf8807b2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8007b60 tx=0x7fbfa8007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.322+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fbfa4083500 con 0x7fbfb806ff20 2026-02-24T18:35:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.322+0000 7fbf93fff640 10 client.4677.objecter ms_dispatch2 0x7fbfb8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.322+0000 7fbf93fff640 3 client.4677.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.322+0000 7fbf93fff640 3 client.4677.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.322+0000 7fbf93fff640 20 client.4677.objecter dump_active .. 0 homeless 2026-02-24T18:35:49.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.423+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbfa4050020 con 0x7fbfb806ff20 2026-02-24T18:35:49.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.423+0000 7fbf93fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:49.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.423+0000 7fbf93fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:49.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.524+0000 7fbfbdbce640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:49.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.524+0000 7fbfbdbce640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:49.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.524+0000 7fbfbdbce640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:49.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.524+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbfb80c7260 con 0x7fbfb806ff20 2026-02-24T18:35:49.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.526+0000 7fbf93fff640 1 -- 10.20.193.18:0/3317336170 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbfa40874c0 con 0x7fbfb806ff20 2026-02-24T18:35:49.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.526+0000 7fbf93fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:49.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.526+0000 7fbf93fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:49.527 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:49.527 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 10 librados: watch_flush enter 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 10 librados: watch_flush exit 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 20 client.4677.objecter shutdown clearing up homeless session... 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 10 client.4677.objecter successfully canceled tick 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 msgr2=0x7fbf8807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 0x7fbf8807b2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8007b60 tx=0x7fbfa8007ab0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 10 monclient: shutdown 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 msgr2=0x7fbfb81f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbfa403a040 tx=0x7fbfa40047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 shutdown_connections 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf88078e00 0x7fbf8807b2c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbfb80c99a0 0x7fbfb806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbfb815a5e0 0x7fbfb81f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 --2- 10.20.193.18:0/3317336170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbfb806ff20 0x7fbfb81f7570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 >> 10.20.193.18:0/3317336170 conn(0x7fbfb80cd780 msgr2=0x7fbfb814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 shutdown_connections 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 -- 10.20.193.18:0/3317336170 wait complete. 2026-02-24T18:35:49.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:49.527+0000 7fbfbdbce640 1 librados: shutdown 2026-02-24T18:35:52.538 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:52.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 10 monclient: get_monmap_and_config 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 10 monclient: build_initial_monmap 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 10 monclient: monmap: 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:52.608232+0000 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:52.608232+0000 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:52.609 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:52.610 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:52.610 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 1 Processor -- start 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 1 -- start start 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.607+0000 7f844fbce640 10 monclient: init 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient: _add_conns ranks=[1,4,5,3,0,2] 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f844806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f84480c99a0 0x7f8448070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-b con 0x7f84480c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f844815a5e0 0x7f844814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-c con 0x7f844815a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84480c7e90 con 0x7f844806ff20 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8448159a40 con 0x7f84480c99a0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f84480c8170 con 0x7f844815a5e0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844fbce640 10 monclient(hunting): authenticate will time out at 1496.289364s 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844d142640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f84480c99a0 0x7f8448070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.608+0000 7f844e144640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f844815a5e0 0x7f844814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f844815a5e0 0x7f844814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:43380/0 (socket says 10.20.193.18:43380) 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 1 -- 10.20.193.18:0/3079344877 learned_addr learned my addr 10.20.193.18:0/3079344877 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): get_auth_request con 0x7f844815a5e0 auth_method 0 2026-02-24T18:35:52.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): get_auth_request con 0x7f84480c99a0 auth_method 0 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.609+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 10 monclient(hunting): handle_auth_done global_id 4625 payload 274 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 1 -- 10.20.193.18:0/3079344877 >> v2:10.20.193.32:3300/0 conn(0x7f84480c99a0 msgr2=0x7f8448070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 1 --2- 10.20.193.18:0/3079344877 >> v2:10.20.193.32:3300/0 conn(0x7f84480c99a0 0x7f8448070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 1 -- 10.20.193.18:0/3079344877 >> v1:10.20.193.32:6789/0 conn(0x7f844806ff20 legacy=0x7f8448159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 10 monclient: _finish_hunting 0 2026-02-24T18:35:52.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 1 monclient: found mon.noname-c 2026-02-24T18:35:52.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:52.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.610+0000 7f844e144640 1 -- 10.20.193.18:0/3079344877 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84480c8450 con 0x7f844815a5e0 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.611+0000 7f844e144640 1 --2- 10.20.193.18:0/3079344877 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844815a5e0 0x7f844814fe10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8444009e50 tx=0x7f844402f2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9ae711690614bc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 1 -- 10.20.193.18:0/3079344877 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844402fd60 con 0x7f844815a5e0 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: dump: 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:52.618 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: _finish_auth 0 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: _check_auth_tickets 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 1 -- 10.20.193.18:0/3079344877 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84440373f0 con 0x7f844815a5e0 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 5 monclient: authenticate success, global_id 4625 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 1 -- 10.20.193.18:0/3079344877 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440377c0 con 0x7f844815a5e0 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:52.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844c941640 10 monclient: dump: 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 4 set_mon_vals no callback set 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 10 monclient: shutdown 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 -- 10.20.193.18:0/3079344877 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844815a5e0 msgr2=0x7f844814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 --2- 10.20.193.18:0/3079344877 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844815a5e0 0x7f844814fe10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8444009e50 tx=0x7f844402f2a0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 -- 10.20.193.18:0/3079344877 shutdown_connections 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 --2- 10.20.193.18:0/3079344877 >> v2:10.20.193.32:3300/0 conn(0x7f84480c99a0 0x7f8448070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 --2- 10.20.193.18:0/3079344877 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844815a5e0 0x7f844814fe10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 -- 10.20.193.18:0/3079344877 >> 10.20.193.18:0/3079344877 conn(0x7f84480cd780 msgr2=0x7f84480cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 -- 10.20.193.18:0/3079344877 shutdown_connections 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.613+0000 7f844fbce640 1 -- 10.20.193.18:0/3079344877 wait complete. 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient: build_initial_monmap 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient: monmap: 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 librados: starting msgr at 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 librados: starting objecter 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 Processor -- start 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 -- start start 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 librados: setting wanted keys 2026-02-24T18:35:52.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 librados: calling monclient init 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient: init 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-c con 0x7f844806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f84480c99a0 0x7f84481f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-a con 0x7f84480c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f844815a5e0 0x7f84481f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): picked mon.noname-b con 0x7f844815a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8448150350 con 0x7f84480c99a0 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84480c8a10 con 0x7f844815a5e0 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f84481504d0 con 0x7f844806ff20 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.614+0000 7f844fbce640 10 monclient(hunting): authenticate will time out at 1496.295778s 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f84480c99a0 0x7f84481f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f844815a5e0 0x7f84481f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:43396/0 (socket says 10.20.193.18:43396) 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 1 -- 10.20.193.18:0/358839089 learned_addr learned my addr 10.20.193.18:0/358839089 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): get_auth_request con 0x7f844806ff20 auth_method 0 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): get_auth_request con 0x7f844815a5e0 auth_method 0 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): get_auth_request con 0x7f84480c99a0 auth_method 0 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d943640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844e144640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.615+0000 7f844d142640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:52.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 10 monclient(hunting): handle_auth_done global_id 4628 payload 824 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 -- 10.20.193.18:0/358839089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f844815a5e0 msgr2=0x7f84481f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f844815a5e0 0x7f84481f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 -- 10.20.193.18:0/358839089 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f84480c99a0 msgr2=0x7f84481f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f84480c99a0 0x7f84481f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 10 monclient: _finish_hunting 0 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 monclient: found mon.noname-c 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.616+0000 7f844d943640 1 -- 10.20.193.18:0/358839089 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844815ae60 con 0x7f844806ff20 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844d943640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f843800c580 tx=0x7f843800ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 client.?.objecter ms_handle_connect 0x7f844806ff20 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438017070 con 0x7f844806ff20 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:52.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 monclient: dump: 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 monclient: _finish_auth 0 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 10 monclient: _check_auth_tickets 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f84367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 5 monclient: authenticate success, global_id 4628 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 10 monclient: _renew_subs 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844815b140 con 0x7f844806ff20 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 10 client.4628.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 10 monclient: _renew_subs 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844815b700 con 0x7f844806ff20 2026-02-24T18:35:52.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.618+0000 7f844fbce640 1 librados: init done 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f844fbce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f844fbce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f844fbce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84481519e0 con 0x7f844806ff20 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84380052d0 con 0x7f844806ff20 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438021060 con 0x7f844806ff20 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f8437fff640 4 set_mon_vals no callback set 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.619+0000 7f84367fc640 10 monclient: dump: 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:52.627 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:52.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.723+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8438024080 con 0x7f844806ff20 2026-02-24T18:35:52.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.724+0000 7f84367fc640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 0x7f841807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:52.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.724+0000 7f844d142640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 0x7f841807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:52.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.724+0000 7f844d142640 10 monclient: get_auth_request con 0x7f8418078e30 auth_method 0 2026-02-24T18:35:52.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.724+0000 7f844d142640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 0x7f841807b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84481ebae0 tx=0x7f843c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:52.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.724+0000 7f84367fc640 10 client.4628.objecter ms_handle_connect 0x7f8418078e30 2026-02-24T18:35:52.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.725+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f843805d570 con 0x7f844806ff20 2026-02-24T18:35:52.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.725+0000 7f84367fc640 10 client.4628.objecter ms_dispatch2 0x7f8448000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:52.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.725+0000 7f84367fc640 3 client.4628.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:52.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.725+0000 7f84367fc640 3 client.4628.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:52.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.725+0000 7f84367fc640 20 client.4628.objecter dump_active .. 0 homeless 2026-02-24T18:35:52.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.731+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8438061530 con 0x7f844806ff20 2026-02-24T18:35:52.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.731+0000 7f84367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:52.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.731+0000 7f84367fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:52.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.831+0000 7f844fbce640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:52.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.831+0000 7f844fbce640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:52.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.831+0000 7f844fbce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:52.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.831+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f844814efe0 con 0x7f844806ff20 2026-02-24T18:35:52.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.835+0000 7f84367fc640 1 -- 10.20.193.18:0/358839089 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8438065830 con 0x7f844806ff20 2026-02-24T18:35:52.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.835+0000 7f84367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:52.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.835+0000 7f84367fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:52.837 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:52.837 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:52.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 10 librados: watch_flush enter 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 10 librados: watch_flush exit 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 20 client.4628.objecter shutdown clearing up homeless session... 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 10 client.4628.objecter successfully canceled tick 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 msgr2=0x7f841807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 0x7f841807b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84481ebae0 tx=0x7f843c009290 comp rx=0 tx=0).stop 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 10 monclient: shutdown 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 msgr2=0x7f844814ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f843800c580 tx=0x7f843800ca50 comp rx=0 tx=0).stop 2026-02-24T18:35:52.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 shutdown_connections 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8418078e30 0x7f841807b2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f844815a5e0 0x7f84481f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f84480c99a0 0x7f84481f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 --2- 10.20.193.18:0/358839089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f844806ff20 0x7f844814ce20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 >> 10.20.193.18:0/358839089 conn(0x7f84480cd780 msgr2=0x7f84480c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 shutdown_connections 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 -- 10.20.193.18:0/358839089 wait complete. 2026-02-24T18:35:52.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:52.836+0000 7f844fbce640 1 librados: shutdown 2026-02-24T18:35:55.847 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 10 monclient: get_monmap_and_config 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 10 monclient: build_initial_monmap 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 10 monclient: monmap: 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:55.915940+0000 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:55.915940+0000 2026-02-24T18:35:55.917 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 1 Processor -- start 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 1 -- start start 2026-02-24T18:35:55.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.915+0000 7f4925197640 10 monclient: init 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient: _add_conns ranks=[4,1,2,0,5,3] 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f492006ff20 0x7f4920070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): picked mon.noname-b con 0x7f492006ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f49200c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f492015a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49200c7e90 con 0x7f49200c99a0 2026-02-24T18:35:55.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49200c8170 con 0x7f492015a5e0 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4920159a40 con 0x7f492006ff20 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f4925197640 10 monclient(hunting): authenticate will time out at 1499.597108s 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491ed76640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f492006ff20 0x7f4920070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491f577640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f492015a5e0 0x7f492014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:54100/0 (socket says 10.20.193.18:54100) 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491f577640 1 -- 10.20.193.18:0/2072288564 learned_addr learned my addr 10.20.193.18:0/2072288564 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491ed76640 10 monclient(hunting): get_auth_request con 0x7f492006ff20 auth_method 0 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:55.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.916+0000 7f491ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.917+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.917+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.917+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 1 -- 10.20.193.18:0/2072288564 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 455023077 0 0) 0x7f49200c8170 con 0x7f492015a5e0 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 10 monclient(hunting): _negotiate 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 10 monclient(hunting): my global_id is 4631 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491dd74640 1 -- 10.20.193.18:0/2072288564 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f490c003620 con 0x7f492015a5e0 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491ed76640 10 monclient(hunting): handle_auth_done global_id 4651 payload 274 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.918+0000 7f491ed76640 1 -- 10.20.193.18:0/2072288564 >> v1:10.20.193.110:6789/0 conn(0x7f492015a5e0 legacy=0x7f492014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 1 -- 10.20.193.18:0/2072288564 >> v1:10.20.193.32:6789/0 conn(0x7f49200c99a0 legacy=0x7f4920159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:55.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:55.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 1 monclient: found mon.noname-b 2026-02-24T18:35:55.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:55.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:35:55.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.919+0000 7f491ed76640 1 -- 10.20.193.18:0/2072288564 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200c8450 con 0x7f492006ff20 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.920+0000 7f491ed76640 1 --2- 10.20.193.18:0/2072288564 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492006ff20 0x7f4920070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49140098d0 tx=0x7f491402eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4d3178cca40965 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 1 -- 10.20.193.18:0/2072288564 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491403c070 con 0x7f492006ff20 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: dump: 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:55.929 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: _finish_auth 0 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: _check_auth_tickets 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 1 -- 10.20.193.18:0/2072288564 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491402fba0 con 0x7f492006ff20 2026-02-24T18:35:55.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 5 monclient: authenticate success, global_id 4651 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 1 -- 10.20.193.18:0/2072288564 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914037440 con 0x7f492006ff20 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 4 set_mon_vals no callback set 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f491dd74640 10 monclient: dump: 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.931 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 10 monclient: shutdown 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 -- 10.20.193.18:0/2072288564 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492006ff20 msgr2=0x7f4920070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 --2- 10.20.193.18:0/2072288564 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492006ff20 0x7f4920070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49140098d0 tx=0x7f491402eed0 comp rx=0 tx=0).stop 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 -- 10.20.193.18:0/2072288564 shutdown_connections 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 --2- 10.20.193.18:0/2072288564 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492006ff20 0x7f4920070300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:55.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 -- 10.20.193.18:0/2072288564 >> 10.20.193.18:0/2072288564 conn(0x7f49200cd780 msgr2=0x7f49200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 -- 10.20.193.18:0/2072288564 shutdown_connections 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.922+0000 7f4925197640 1 -- 10.20.193.18:0/2072288564 wait complete. 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 10 monclient: build_initial_monmap 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 10 monclient: monmap: 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:55.933 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 librados: starting msgr at 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 librados: starting objecter 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 Processor -- start 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 -- start start 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 librados: setting wanted keys 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 1 librados: calling monclient init 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.923+0000 7f4925197640 10 monclient: init 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 0x7f49201f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:55.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): picked mon.noname-c con 0x7f492006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 0x7f49200c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): picked mon.noname-a con 0x7f49200c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492015a5e0 0x7f49201f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): picked mon.noname-b con 0x7f492015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4920150520 con 0x7f49200c99a0 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49200c8a10 con 0x7f492015a5e0 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49201506a0 con 0x7f492006ff20 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:55.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f4925197640 10 monclient(hunting): authenticate will time out at 1499.605215s 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 0x7f49200c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491ed76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 0x7f49201f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 0x7f49200c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49700/0 (socket says 10.20.193.18:49700) 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491f577640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492015a5e0 0x7f49201f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 1 -- 10.20.193.18:0/3373894573 learned_addr learned my addr 10.20.193.18:0/3373894573 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): get_auth_request con 0x7f49200c99a0 auth_method 0 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491ed76640 10 monclient(hunting): get_auth_request con 0x7f492006ff20 auth_method 0 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:55.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491f577640 10 monclient(hunting): get_auth_request con 0x7f492015a5e0 auth_method 0 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491f577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491f577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491f577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.924+0000 7f491e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491f577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:55.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491f577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.925+0000 7f491f577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 10 monclient(hunting): handle_auth_done global_id 4634 payload 824 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 -- 10.20.193.18:0/3373894573 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492015a5e0 msgr2=0x7f49201f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492015a5e0 0x7f49201f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 -- 10.20.193.18:0/3373894573 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 msgr2=0x7f49200c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 0x7f49200c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 10 monclient: _finish_hunting 0 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 monclient: found mon.noname-c 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.926+0000 7f491ed76640 1 -- 10.20.193.18:0/3373894573 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4920155720 con 0x7f492006ff20 2026-02-24T18:35:55.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.927+0000 7f491ed76640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 0x7f49201f4ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4920072430 tx=0x7f49140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:55.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.927+0000 7f48ff7fe640 10 client.?.objecter ms_handle_connect 0x7f492006ff20 2026-02-24T18:35:55.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.927+0000 7f48ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:55.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491403c040 con 0x7f492006ff20 2026-02-24T18:35:55.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:55.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:35:55.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: dump: 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4914004250 con 0x7f492006ff20 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 5 monclient: authenticate success, global_id 4634 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 10 monclient: _renew_subs 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4920155a00 con 0x7f492006ff20 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914003710 con 0x7f492006ff20 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:35:55.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f48ff7fe640 10 monclient: dump: 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f491d573640 4 set_mon_vals no callback set 2026-02-24T18:35:55.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 10 client.4634.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 10 monclient: _renew_subs 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4920155fc0 con 0x7f492006ff20 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.929+0000 7f4925197640 1 librados: init done 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.930+0000 7f4925197640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.930+0000 7f4925197640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.930+0000 7f4925197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:55.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.930+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4920070300 con 0x7f492006ff20 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.941+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f49140038b0 con 0x7f492006ff20 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.941+0000 7f48ff7fe640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 0x7f48ec07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.942+0000 7f491e575640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 0x7f48ec07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.942+0000 7f491e575640 10 monclient: get_auth_request con 0x7f48ec078e00 auth_method 0 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.942+0000 7f48ff7fe640 10 client.4634.objecter ms_handle_connect 0x7f48ec078e00 2026-02-24T18:35:55.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:55.942+0000 7f491e575640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 0x7f48ec07b2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49200c43b0 tx=0x7f4908002a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:56.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.037+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f491402fa60 con 0x7f492006ff20 2026-02-24T18:35:56.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.037+0000 7f48ff7fe640 10 client.4634.objecter ms_dispatch2 0x7f4920000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:56.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.037+0000 7f48ff7fe640 3 client.4634.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:56.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.037+0000 7f48ff7fe640 3 client.4634.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:56.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.037+0000 7f48ff7fe640 20 client.4634.objecter dump_active .. 0 homeless 2026-02-24T18:35:56.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.043+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f49140867e0 con 0x7f492006ff20 2026-02-24T18:35:56.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.043+0000 7f48ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:56.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.043+0000 7f48ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:56.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.146+0000 7f4925197640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:56.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.146+0000 7f4925197640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:56.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.146+0000 7f4925197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:35:56.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.146+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f49201573f0 con 0x7f492006ff20 2026-02-24T18:35:56.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.149+0000 7f48ff7fe640 1 -- 10.20.193.18:0/3373894573 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f491408aae0 con 0x7f492006ff20 2026-02-24T18:35:56.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.149+0000 7f48ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:56.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.149+0000 7f48ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:56.150 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:56.150 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:56.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 10 librados: watch_flush enter 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 10 librados: watch_flush exit 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 20 client.4634.objecter shutdown clearing up homeless session... 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 10 client.4634.objecter successfully canceled tick 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 msgr2=0x7f48ec07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 0x7f48ec07b2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49200c43b0 tx=0x7f4908002a30 comp rx=0 tx=0).stop 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 10 monclient: shutdown 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 msgr2=0x7f49201f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 0x7f49201f4ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4920072430 tx=0x7f49140047c0 comp rx=0 tx=0).stop 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 shutdown_connections 2026-02-24T18:35:56.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f48ec078e00 0x7f48ec07b2c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f492015a5e0 0x7f49201f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49200c99a0 0x7f49200c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 --2- 10.20.193.18:0/3373894573 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f492006ff20 0x7f49201f4ce0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 >> 10.20.193.18:0/3373894573 conn(0x7f49200cd780 msgr2=0x7f492014e600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 shutdown_connections 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 -- 10.20.193.18:0/3373894573 wait complete. 2026-02-24T18:35:56.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:56.150+0000 7f4925197640 1 librados: shutdown 2026-02-24T18:35:59.163 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:59.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.232+0000 7f326f1c3640 10 monclient: get_monmap_and_config 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.232+0000 7f326f1c3640 10 monclient: build_initial_monmap 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.232+0000 7f326f1c3640 10 monclient: monmap: 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:35:59.233568+0000 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:35:59.233568+0000 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:35:59.235 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 Processor -- start 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 -- start start 2026-02-24T18:35:59.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient: init 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient: _add_conns ranks=[3,1,4,2,5,0] 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f326806ff20 0x7f3268070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-a con 0x7f326806ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f32680c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f326815a5e0 0x7f326814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-b con 0x7f326815a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32680c7e90 con 0x7f32680c99a0 2026-02-24T18:35:59.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3268159a40 con 0x7f326806ff20 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f32680c8170 con 0x7f326815a5e0 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.233+0000 7f326f1c3640 10 monclient(hunting): authenticate will time out at 1502.914756s 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f326806ff20 0x7f3268070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f326806ff20 0x7f3268070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45758/0 (socket says 10.20.193.18:45758) 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 1 -- 10.20.193.18:0/154556233 learned_addr learned my addr 10.20.193.18:0/154556233 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): get_auth_request con 0x7f326806ff20 auth_method 0 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:59.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326d739640 1 --2- 10.20.193.18:0/154556233 >> v2:10.20.193.32:3300/0 conn(0x7f326815a5e0 0x7f326814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326d739640 10 monclient(hunting): get_auth_request con 0x7f326815a5e0 auth_method 0 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326d739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326d739640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.234+0000 7f326d739640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:59.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 10 monclient(hunting): handle_auth_done global_id 4686 payload 274 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 1 -- 10.20.193.18:0/154556233 >> v2:10.20.193.32:3300/0 conn(0x7f326815a5e0 msgr2=0x7f326814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 1 --2- 10.20.193.18:0/154556233 >> v2:10.20.193.32:3300/0 conn(0x7f326815a5e0 0x7f326814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 1 -- 10.20.193.18:0/154556233 >> v1:10.20.193.32:6789/0 conn(0x7f32680c99a0 legacy=0x7f3268159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 10 monclient: _finish_hunting 0 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 1 monclient: found mon.noname-a 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.235+0000 7f326cf38640 1 -- 10.20.193.18:0/154556233 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32680c8450 con 0x7f326806ff20 2026-02-24T18:35:59.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.236+0000 7f326cf38640 1 --2- 10.20.193.18:0/154556233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f326806ff20 0x7f3268070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3258009970 tx=0x7f325802eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370cd5007ab58f69 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:59.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 1 -- 10.20.193.18:0/154556233 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325803c070 con 0x7f326806ff20 2026-02-24T18:35:59.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:59.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:59.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: dump: 2026-02-24T18:35:59.249 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:35:59.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 5 monclient: authenticate success, global_id 4686 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 1 -- 10.20.193.18:0/154556233 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325802fbc0 con 0x7f326806ff20 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 1 -- 10.20.193.18:0/154556233 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258037440 con 0x7f326806ff20 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 10 monclient: get_monmap_and_config success 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 4 set_mon_vals no callback set 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f325f7fe640 10 monclient: dump: 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:59.251 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 10 monclient: shutdown 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 -- 10.20.193.18:0/154556233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f326806ff20 msgr2=0x7f3268070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 --2- 10.20.193.18:0/154556233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f326806ff20 0x7f3268070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3258009970 tx=0x7f325802eef0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 -- 10.20.193.18:0/154556233 shutdown_connections 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 --2- 10.20.193.18:0/154556233 >> v2:10.20.193.32:3300/0 conn(0x7f326815a5e0 0x7f326814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 --2- 10.20.193.18:0/154556233 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f326806ff20 0x7f3268070300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 -- 10.20.193.18:0/154556233 >> 10.20.193.18:0/154556233 conn(0x7f32680cd780 msgr2=0x7f32680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 -- 10.20.193.18:0/154556233 shutdown_connections 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.238+0000 7f326f1c3640 1 -- 10.20.193.18:0/154556233 wait complete. 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient: build_initial_monmap 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient: monmap: 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:35:59.253 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 librados: starting msgr at 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 librados: starting objecter 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 Processor -- start 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 -- start start 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 librados: setting wanted keys 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 librados: calling monclient init 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient: init 2026-02-24T18:35:59.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient: _reopen_session rank -1 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f326806ff20 0x7f32681f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-b con 0x7f326806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-a con 0x7f32680c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f326815a5e0 0x7f3268072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): picked mon.noname-c con 0x7f326815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3268150350 con 0x7f32680c99a0 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32680c8a10 con 0x7f326806ff20 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:35:59.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f32681504d0 con 0x7f326815a5e0 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): _renew_subs 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.239+0000 7f326f1c3640 10 monclient(hunting): authenticate will time out at 1502.920994s 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45764/0 (socket says 10.20.193.18:45764) 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 1 -- 10.20.193.18:0/929361626 learned_addr learned my addr 10.20.193.18:0/929361626 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): get_auth_request con 0x7f32680c99a0 auth_method 0 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f326806ff20 0x7f32681f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f325ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): get_auth_request con 0x7f326806ff20 auth_method 0 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326d739640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f326815a5e0 0x7f3268072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326cf38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326d739640 10 monclient(hunting): get_auth_request con 0x7f326815a5e0 auth_method 0 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326d739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:35:59.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326d739640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.240+0000 7f326d739640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.241+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.241+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.241+0000 7f326d739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.241+0000 7f325ffff640 10 monclient(hunting): handle_auth_done global_id 4689 payload 824 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 -- 10.20.193.18:0/929361626 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f326815a5e0 msgr2=0x7f3268072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f326815a5e0 0x7f3268072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 -- 10.20.193.18:0/929361626 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f326806ff20 msgr2=0x7f32681f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f326806ff20 0x7f32681f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 monclient: found mon.noname-a 2026-02-24T18:35:59.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 -- 10.20.193.18:0/929361626 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32680c10d0 con 0x7f32680c99a0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325ffff640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f325000c580 tx=0x7f325000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325d7fa640 10 client.?.objecter ms_handle_connect 0x7f32680c99a0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.242+0000 7f325d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250017070 con 0x7f32680c99a0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: dump: 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:59.259 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32500052d0 con 0x7f32680c99a0 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:35:59.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32500036b0 con 0x7f32680c99a0 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325d7fa640 10 monclient: dump: 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:35:59.261 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f325effd640 4 set_mon_vals no callback set 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 5 monclient: authenticate success, global_id 4689 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 10 monclient: _renew_subs 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32681f3c70 con 0x7f32680c99a0 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 10 client.4689.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 10 monclient: _renew_subs 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681f4220 con 0x7f32680c99a0 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.244+0000 7f326f1c3640 1 librados: init done 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.245+0000 7f326f1c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:59.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.245+0000 7f326f1c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:59.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.245+0000 7f326f1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.245+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3268150860 con 0x7f32680c99a0 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.256+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3250003850 con 0x7f32680c99a0 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.257+0000 7f325d7fa640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 0x7f323807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.257+0000 7f326cf38640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 0x7f323807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.257+0000 7f326cf38640 10 monclient: get_auth_request con 0x7f3238078e00 auth_method 0 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.258+0000 7f326cf38640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 0x7f323807b2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f32680c5350 tx=0x7f3258039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:35:59.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.258+0000 7f325d7fa640 10 client.4689.objecter ms_handle_connect 0x7f3238078e00 2026-02-24T18:35:59.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.265+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f325005d470 con 0x7f32680c99a0 2026-02-24T18:35:59.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.265+0000 7f325d7fa640 10 client.4689.objecter ms_dispatch2 0x7f3268000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:35:59.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.265+0000 7f325d7fa640 3 client.4689.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:35:59.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.265+0000 7f325d7fa640 3 client.4689.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:35:59.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.265+0000 7f325d7fa640 20 client.4689.objecter dump_active .. 0 homeless 2026-02-24T18:35:59.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.373+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3250021020 con 0x7f32680c99a0 2026-02-24T18:35:59.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.373+0000 7f325d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:35:59.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.373+0000 7f325d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:35:59.401 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:37.347 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.403 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:37.349 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.411 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:37.357 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.430 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:37.376 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.431 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:37.377 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.432 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:37.340 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.433 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:37.341 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.440 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:37.385 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.441 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:37.348 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:35:59.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.479+0000 7f326f1c3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:35:59.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.479+0000 7f326f1c3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:59.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.479+0000 7f326f1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:35:59.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.479+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f326814f510 con 0x7f32680c99a0 2026-02-24T18:35:59.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.484+0000 7f325d7fa640 1 -- 10.20.193.18:0/929361626 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3250061430 con 0x7f32680c99a0 2026-02-24T18:35:59.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.484+0000 7f325d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:35:59.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.484+0000 7f325d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:35:59.487 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:35:59.487 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 10 librados: watch_flush enter 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 10 librados: watch_flush exit 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 20 client.4689.objecter shutdown clearing up homeless session... 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 10 client.4689.objecter successfully canceled tick 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 msgr2=0x7f323807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 0x7f323807b2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f32680c5350 tx=0x7f3258039040 comp rx=0 tx=0).stop 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 10 monclient: shutdown 2026-02-24T18:35:59.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 msgr2=0x7f32681f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f325000c580 tx=0x7f325000ca50 comp rx=0 tx=0).stop 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 shutdown_connections 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3238078e00 0x7f323807b2c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f326815a5e0 0x7f3268072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32680c99a0 0x7f32681f3730 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 --2- 10.20.193.18:0/929361626 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f326806ff20 0x7f32681f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 >> 10.20.193.18:0/929361626 conn(0x7f32680cd780 msgr2=0x7f326814e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 shutdown_connections 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 -- 10.20.193.18:0/929361626 wait complete. 2026-02-24T18:35:59.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:35:59.486+0000 7f326f1c3640 1 librados: shutdown 2026-02-24T18:36:02.499 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:02.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.567+0000 7f9568057640 10 monclient: get_monmap_and_config 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.567+0000 7f9568057640 10 monclient: build_initial_monmap 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.567+0000 7f9568057640 10 monclient: monmap: 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:02.568649+0000 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:02.568649+0000 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:02.570 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 Processor -- start 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 -- start start 2026-02-24T18:36:02.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient: init 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient: _add_conns ranks=[2,1,4,5,3,0] 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95601527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9560152bb0 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f95600c6190 0x7f95600c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:02.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): picked mon.noname-b con 0x7f95600c6190 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956006e300 con 0x7f9560152bb0 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956006e5e0 con 0x7f95601527d0 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f956006f1f0 con 0x7f95600c6190 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.568+0000 7f9568057640 10 monclient(hunting): authenticate will time out at 1506.249774s 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f95665cd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f95600c6190 0x7f95600c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f9565dcc640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f95601527d0 0x7f956006da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:44980/0 (socket says 10.20.193.18:44980) 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f9565dcc640 1 -- 10.20.193.18:0/164744234 learned_addr learned my addr 10.20.193.18:0/164744234 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f95665cd640 10 monclient(hunting): get_auth_request con 0x7f95600c6190 auth_method 0 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f95665cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:02.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f95665cd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.569+0000 7f95665cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 1 -- 10.20.193.18:0/164744234 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272702092 0 0) 0x7f956006e5e0 con 0x7f95601527d0 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 10 monclient(hunting): _negotiate 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 10 monclient(hunting): my global_id is 4640 2026-02-24T18:36:02.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.570+0000 7f9564dca640 1 -- 10.20.193.18:0/164744234 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9548003620 con 0x7f95601527d0 2026-02-24T18:36:02.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 10 monclient(hunting): handle_auth_done global_id 4669 payload 274 2026-02-24T18:36:02.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 1 -- 10.20.193.18:0/164744234 >> v1:10.20.193.110:6789/0 conn(0x7f95601527d0 legacy=0x7f956006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 1 -- 10.20.193.18:0/164744234 >> v1:10.20.193.32:6789/0 conn(0x7f9560152bb0 legacy=0x7f95600c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 10 monclient: _finish_hunting 0 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 1 monclient: found mon.noname-b 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.572+0000 7f95665cd640 1 -- 10.20.193.18:0/164744234 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956006e8c0 con 0x7f95600c6190 2026-02-24T18:36:02.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.574+0000 7f95665cd640 1 --2- 10.20.193.18:0/164744234 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95600c8e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f956006f370 tx=0x7f955c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a3c6d030149208 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:02.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 1 -- 10.20.193.18:0/164744234 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c03c070 con 0x7f95600c6190 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: dump: 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.583 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: _finish_auth 0 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: _check_auth_tickets 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 1 -- 10.20.193.18:0/164744234 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955c037440 con 0x7f95600c6190 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 5 monclient: authenticate success, global_id 4669 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 1 -- 10.20.193.18:0/164744234 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c037810 con 0x7f95600c6190 2026-02-24T18:36:02.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 4 set_mon_vals no callback set 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9564dca640 10 monclient: dump: 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:02.585 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 10 monclient: shutdown 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 -- 10.20.193.18:0/164744234 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 msgr2=0x7f95600c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 --2- 10.20.193.18:0/164744234 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95600c8e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f956006f370 tx=0x7f955c02f230 comp rx=0 tx=0).stop 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 -- 10.20.193.18:0/164744234 shutdown_connections 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 --2- 10.20.193.18:0/164744234 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95600c8e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 -- 10.20.193.18:0/164744234 >> 10.20.193.18:0/164744234 conn(0x7f95600cd780 msgr2=0x7f95600cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 -- 10.20.193.18:0/164744234 shutdown_connections 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.576+0000 7f9568057640 1 -- 10.20.193.18:0/164744234 wait complete. 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient: build_initial_monmap 2026-02-24T18:36:02.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient: monmap: 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 librados: starting msgr at 2026-02-24T18:36:02.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 librados: starting objecter 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 Processor -- start 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 -- start start 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 librados: setting wanted keys 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 librados: calling monclient init 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient: init 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95601f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 10 monclient(hunting): picked mon.noname-b con 0x7f95600c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.577+0000 7f9568057640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f95601527d0 0x7f9560153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): picked mon.noname-c con 0x7f95601527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 0x7f9560072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): picked mon.noname-a con 0x7f9560152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:02.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f95600c93a0 con 0x7f9560152bb0 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f956006ee80 con 0x7f95600c6190 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95600c9520 con 0x7f95601527d0 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9568057640 10 monclient(hunting): authenticate will time out at 1506.259027s 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 0x7f9560072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95601f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f95601527d0 0x7f9560153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 0x7f9560072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45774/0 (socket says 10.20.193.18:45774) 2026-02-24T18:36:02.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 1 -- 10.20.193.18:0/452895824 learned_addr learned my addr 10.20.193.18:0/452895824 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): get_auth_request con 0x7f9560152bb0 auth_method 0 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): get_auth_request con 0x7f95600c6190 auth_method 0 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): get_auth_request con 0x7f95601527d0 auth_method 0 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:02.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f9565dcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95665cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.578+0000 7f95655cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 10 monclient(hunting): handle_auth_done global_id 4675 payload 824 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 -- 10.20.193.18:0/452895824 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f95601527d0 msgr2=0x7f9560153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f95601527d0 0x7f9560153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 -- 10.20.193.18:0/452895824 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 msgr2=0x7f9560072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 0x7f9560072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 10 monclient: _finish_hunting 0 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 monclient: found mon.noname-b 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.580+0000 7f9565dcc640 1 -- 10.20.193.18:0/452895824 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95601f4390 con 0x7f95600c6190 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.581+0000 7f9565dcc640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95601f3d30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f955000c4d0 tx=0x7f955000c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.581+0000 7f954e7fc640 10 client.?.objecter ms_handle_connect 0x7f95600c6190 2026-02-24T18:36:02.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.581+0000 7f954e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:02.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550017070 con 0x7f95600c6190 2026-02-24T18:36:02.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:02.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:02.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: dump: 2026-02-24T18:36:02.594 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:02.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 5 monclient: authenticate success, global_id 4675 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 10 monclient: _renew_subs 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95601f4670 con 0x7f95600c6190 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 10 client.4675.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 10 monclient: _renew_subs 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9560072520 con 0x7f95600c6190 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f9568057640 1 librados: init done 2026-02-24T18:36:02.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9550005de0 con 0x7f95600c6190 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550021060 con 0x7f95600c6190 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954e7fc640 10 monclient: dump: 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:02.597 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.582+0000 7f954ffff640 4 set_mon_vals no callback set 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.583+0000 7f9568057640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.583+0000 7f9568057640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.583+0000 7f9568057640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.583+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95600c8e60 con 0x7f95600c6190 2026-02-24T18:36:02.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.594+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9550024080 con 0x7f95600c6190 2026-02-24T18:36:02.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.594+0000 7f954e7fc640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 0x7f953407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.595+0000 7f95655cb640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 0x7f953407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.595+0000 7f95655cb640 10 monclient: get_auth_request con 0x7f9534078e00 auth_method 0 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.595+0000 7f954e7fc640 10 client.4675.objecter ms_handle_connect 0x7f9534078e00 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.595+0000 7f95655cb640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 0x7f953407b2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9554002410 tx=0x7f9554007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.596+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f955005d4e0 con 0x7f95600c6190 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.596+0000 7f954e7fc640 10 client.4675.objecter ms_dispatch2 0x7f9560000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.596+0000 7f954e7fc640 3 client.4675.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.596+0000 7f954e7fc640 3 client.4675.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:02.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.596+0000 7f954e7fc640 20 client.4675.objecter dump_active .. 0 homeless 2026-02-24T18:36:02.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.695+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f95500614a0 con 0x7f95600c6190 2026-02-24T18:36:02.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.695+0000 7f954e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:02.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.695+0000 7f954e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:02.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.796+0000 7f9568057640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:02.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.796+0000 7f9568057640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:02.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.796+0000 7f9568057640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:02.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.796+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9560141570 con 0x7f95600c6190 2026-02-24T18:36:02.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.799+0000 7f954e7fc640 1 -- 10.20.193.18:0/452895824 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f95500657a0 con 0x7f95600c6190 2026-02-24T18:36:02.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.799+0000 7f954e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:02.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.799+0000 7f954e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:02.800 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:02.800 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:02.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 10 librados: watch_flush enter 2026-02-24T18:36:02.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 10 librados: watch_flush exit 2026-02-24T18:36:02.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 20 client.4675.objecter shutdown clearing up homeless session... 2026-02-24T18:36:02.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 10 client.4675.objecter successfully canceled tick 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 msgr2=0x7f953407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 0x7f953407b2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9554002410 tx=0x7f9554007ef0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 10 monclient: shutdown 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 msgr2=0x7f95601f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95601f3d30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f955000c4d0 tx=0x7f955000c9a0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 shutdown_connections 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9534078e00 0x7f953407b2c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9560152bb0 0x7f9560072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f95601527d0 0x7f9560153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 --2- 10.20.193.18:0/452895824 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f95600c6190 0x7f95601f3d30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:02.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 >> 10.20.193.18:0/452895824 conn(0x7f95600cd780 msgr2=0x7f95601501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:02.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 shutdown_connections 2026-02-24T18:36:02.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 -- 10.20.193.18:0/452895824 wait complete. 2026-02-24T18:36:02.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:02.800+0000 7f9568057640 1 librados: shutdown 2026-02-24T18:36:05.812 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:05.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.879+0000 7f92f8205640 10 monclient: get_monmap_and_config 2026-02-24T18:36:05.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.879+0000 7f92f8205640 10 monclient: build_initial_monmap 2026-02-24T18:36:05.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.879+0000 7f92f8205640 10 monclient: monmap: 2026-02-24T18:36:05.881 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:05.880684+0000 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:05.880684+0000 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:05.882 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:05.883 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 Processor -- start 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 -- start start 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient: init 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient: _add_conns ranks=[5,1,0,2,3,4] 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f92f00c90d0 0x7f92f00c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-c con 0x7f92f00c90d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f92f0072bf0 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92f0073040 addr v1:10.20.193.18:6789/0 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f0159440 con 0x7f92f0073040 2026-02-24T18:36:05.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f0159720 con 0x7f92f0072bf0 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f92f015ce70 con 0x7f92f00c90d0 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.880+0000 7f92f8205640 10 monclient(hunting): authenticate will time out at 1509.561997s 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f677b640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f92f0073040 0x7f92f0074250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:33756/0 (socket says 10.20.193.18:33756) 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f677b640 1 -- 10.20.193.18:0/2119330089 learned_addr learned my addr 10.20.193.18:0/2119330089 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f5f7a640 1 --2- 10.20.193.18:0/2119330089 >> v2:10.20.193.110:3300/0 conn(0x7f92f00c90d0 0x7f92f00c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f5f7a640 10 monclient(hunting): get_auth_request con 0x7f92f00c90d0 auth_method 0 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f5f7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:05.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f5f7a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.881+0000 7f92f5f7a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706095199 0 0) 0x7f92f0159440 con 0x7f92f0073040 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 10 monclient(hunting): _negotiate 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 10 monclient(hunting): my global_id is 4695 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.882+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d8003620 con 0x7f92f0073040 2026-02-24T18:36:05.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174194059 0 0) 0x7f92f0159720 con 0x7f92f0072bf0 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 10 monclient(hunting): _negotiate 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 10 monclient(hunting): my global_id is 4678 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f0159440 con 0x7f92f0072bf0 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_done global_id 4646 payload 274 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 1 -- 10.20.193.18:0/2119330089 >> v1:10.20.193.32:6789/0 conn(0x7f92f0072bf0 legacy=0x7f92f0158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 1 -- 10.20.193.18:0/2119330089 >> v1:10.20.193.18:6789/0 conn(0x7f92f0073040 legacy=0x7f92f0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 10 monclient: _finish_hunting 0 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 1 monclient: found mon.noname-c 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:05.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:36:05.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.883+0000 7f92f5f7a640 1 -- 10.20.193.18:0/2119330089 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f0073790 con 0x7f92f00c90d0 2026-02-24T18:36:05.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.884+0000 7f92f5f7a640 1 --2- 10.20.193.18:0/2119330089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f00c94b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098a0 tx=0x7f92e002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce27abf6647a3cb5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:05.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e003c070 con 0x7f92f00c90d0 2026-02-24T18:36:05.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: dump: 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:05.897 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: _finish_auth 0 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: _check_auth_tickets 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e002fb70 con 0x7f92f00c90d0 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 1 -- 10.20.193.18:0/2119330089 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0037440 con 0x7f92f00c90d0 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 5 monclient: authenticate success, global_id 4646 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 4 set_mon_vals no callback set 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:36:05.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f4f78640 10 monclient: dump: 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 10 monclient: shutdown 2026-02-24T18:36:05.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 -- 10.20.193.18:0/2119330089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 msgr2=0x7f92f00c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 --2- 10.20.193.18:0/2119330089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f00c94b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098a0 tx=0x7f92e002ee60 comp rx=0 tx=0).stop 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 -- 10.20.193.18:0/2119330089 shutdown_connections 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 --2- 10.20.193.18:0/2119330089 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f00c94b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 -- 10.20.193.18:0/2119330089 >> 10.20.193.18:0/2119330089 conn(0x7f92f00cd780 msgr2=0x7f92f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 -- 10.20.193.18:0/2119330089 shutdown_connections 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.886+0000 7f92f8205640 1 -- 10.20.193.18:0/2119330089 wait complete. 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient: build_initial_monmap 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient: monmap: 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:05.900 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 librados: starting msgr at 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 librados: starting objecter 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 Processor -- start 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 -- start start 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 librados: setting wanted keys 2026-02-24T18:36:05.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 librados: calling monclient init 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient: init 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f92f0072bf0 0x7f92f00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-b con 0x7f92f0072bf0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-a con 0x7f92f0073040 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f01f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): picked mon.noname-c con 0x7f92f00c90d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0073d50 con 0x7f92f0073040 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:05.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0073540 con 0x7f92f0072bf0 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0159060 con 0x7f92f00c90d0 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.887+0000 7f92f8205640 10 monclient(hunting): authenticate will time out at 1509.568833s 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45788/0 (socket says 10.20.193.18:45788) 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 1 -- 10.20.193.18:0/2583082484 learned_addr learned my addr 10.20.193.18:0/2583082484 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): get_auth_request con 0x7f92f0073040 auth_method 0 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f92f0072bf0 0x7f92f00c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f01f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5779640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): get_auth_request con 0x7f92f0072bf0 auth_method 0 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): get_auth_request con 0x7f92f00c90d0 auth_method 0 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:05.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f5f7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.888+0000 7f92f677b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 10 monclient(hunting): handle_auth_done global_id 4698 payload 824 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 -- 10.20.193.18:0/2583082484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 msgr2=0x7f92f01f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f01f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 -- 10.20.193.18:0/2583082484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f92f0072bf0 msgr2=0x7f92f00c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f92f0072bf0 0x7f92f00c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 10 monclient: _finish_hunting 0 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 monclient: found mon.noname-a 2026-02-24T18:36:05.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.889+0000 7f92f5779640 1 -- 10.20.193.18:0/2583082484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f014ca00 con 0x7f92f0073040 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.890+0000 7f92f5779640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c520 tx=0x7f92e400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.890+0000 7f92de7fc640 10 client.?.objecter ms_handle_connect 0x7f92f0073040 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.890+0000 7f92de7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.892+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e40052d0 con 0x7f92f0073040 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.892+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.892+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.892+0000 7f92de7fc640 10 monclient: dump: 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.906 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e4005470 con 0x7f92f0073040 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4003910 con 0x7f92f0073040 2026-02-24T18:36:05.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92dffff640 4 set_mon_vals no callback set 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92de7fc640 10 monclient: dump: 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:05.908 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 5 monclient: authenticate success, global_id 4698 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _renew_subs 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01f3d00 con 0x7f92f0073040 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 client.4698.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _renew_subs 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01f42b0 con 0x7f92f0073040 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 1 librados: init done 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:05.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:05.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.893+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f00c94b0 con 0x7f92f0073040 2026-02-24T18:36:06.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.998+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f92e4003ab0 con 0x7f92f0073040 2026-02-24T18:36:06.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.999+0000 7f92de7fc640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 0x7f92c007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:06.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.999+0000 7f92f5f7a640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 0x7f92c007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:06.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:05.999+0000 7f92f5f7a640 10 monclient: get_auth_request con 0x7f92c0078e00 auth_method 0 2026-02-24T18:36:06.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.000+0000 7f92de7fc640 10 client.4698.objecter ms_handle_connect 0x7f92c0078e00 2026-02-24T18:36:06.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.000+0000 7f92f5f7a640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 0x7f92c007b2c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92e0039000 tx=0x7f92e0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f92e405e020 con 0x7f92f0073040 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 10 client.4698.objecter ms_dispatch2 0x7f92f0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 3 client.4698.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 3 client.4698.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 20 client.4698.objecter dump_active .. 0 homeless 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f92e4061fe0 con 0x7f92f0073040 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:06.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.006+0000 7f92de7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:06.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.109+0000 7f92f8205640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:06.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.109+0000 7f92f8205640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:06.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.109+0000 7f92f8205640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:06.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.109+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f92f0156300 con 0x7f92f0073040 2026-02-24T18:36:06.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.111+0000 7f92de7fc640 1 -- 10.20.193.18:0/2583082484 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f92e40662e0 con 0x7f92f0073040 2026-02-24T18:36:06.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.111+0000 7f92de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:06.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.111+0000 7f92de7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:06.113 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:06.113 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 10 librados: watch_flush enter 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 10 librados: watch_flush exit 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 20 client.4698.objecter shutdown clearing up homeless session... 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 10 client.4698.objecter successfully canceled tick 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 msgr2=0x7f92c007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 0x7f92c007b2c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92e0039000 tx=0x7f92e0033000 comp rx=0 tx=0).stop 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 10 monclient: shutdown 2026-02-24T18:36:06.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 msgr2=0x7f92f01f76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c520 tx=0x7f92e400c9f0 comp rx=0 tx=0).stop 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 shutdown_connections 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92c0078e00 0x7f92c007b2c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f92f00c90d0 0x7f92f01f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f92f0073040 0x7f92f01f76d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 --2- 10.20.193.18:0/2583082484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f92f0072bf0 0x7f92f00c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 >> 10.20.193.18:0/2583082484 conn(0x7f92f00cd780 msgr2=0x7f92f006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 shutdown_connections 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 -- 10.20.193.18:0/2583082484 wait complete. 2026-02-24T18:36:06.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:06.113+0000 7f92f8205640 1 librados: shutdown 2026-02-24T18:36:09.126 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:09.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 10 monclient: get_monmap_and_config 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 10 monclient: build_initial_monmap 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 10 monclient: monmap: 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:09.198000+0000 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:09.198000+0000 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.199 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:09.200 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 1 Processor -- start 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 1 -- start start 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.196+0000 7f7cf86da640 10 monclient: init 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient: _add_conns ranks=[3,1,4,5,0,2] 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7cf006ff20 0x7f7cf0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-a con 0x7f7cf006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:09.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7cf00c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7cf015a5e0 0x7f7cf014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-b con 0x7f7cf015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf00c7e90 con 0x7f7cf00c99a0 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf0159a40 con 0x7f7cf006ff20 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf00c8170 con 0x7f7cf015a5e0 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf86da640 10 monclient(hunting): authenticate will time out at 1512.879233s 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7cf006ff20 0x7f7cf0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7cf006ff20 0x7f7cf0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47136/0 (socket says 10.20.193.18:47136) 2026-02-24T18:36:09.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 1 -- 10.20.193.18:0/131130490 learned_addr learned my addr 10.20.193.18:0/131130490 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 10 monclient(hunting): get_auth_request con 0x7f7cf006ff20 auth_method 0 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.197+0000 7f7cf644f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf6c50640 1 --2- 10.20.193.18:0/131130490 >> v2:10.20.193.32:3300/0 conn(0x7f7cf015a5e0 0x7f7cf014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf6c50640 10 monclient(hunting): get_auth_request con 0x7f7cf015a5e0 auth_method 0 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf6c50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf6c50640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:09.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.198+0000 7f7cf6c50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 10 monclient(hunting): handle_auth_done global_id 4701 payload 274 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 1 -- 10.20.193.18:0/131130490 >> v2:10.20.193.32:3300/0 conn(0x7f7cf015a5e0 msgr2=0x7f7cf014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 1 --2- 10.20.193.18:0/131130490 >> v2:10.20.193.32:3300/0 conn(0x7f7cf015a5e0 0x7f7cf014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 1 -- 10.20.193.18:0/131130490 >> v1:10.20.193.32:6789/0 conn(0x7f7cf00c99a0 legacy=0x7f7cf0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 10 monclient: _finish_hunting 0 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 1 monclient: found mon.noname-a 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.199+0000 7f7cf644f640 1 -- 10.20.193.18:0/131130490 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf00c8450 con 0x7f7cf006ff20 2026-02-24T18:36:09.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.200+0000 7f7cf644f640 1 --2- 10.20.193.18:0/131130490 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf006ff20 0x7f7cf0070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009970 tx=0x7f7ce002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840648a5e100034e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:09.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 1 -- 10.20.193.18:0/131130490 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce003c070 con 0x7f7cf006ff20 2026-02-24T18:36:09.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: dump: 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:09.213 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: _finish_auth 0 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: _check_auth_tickets 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 5 monclient: authenticate success, global_id 4701 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 1 -- 10.20.193.18:0/131130490 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce002fbc0 con 0x7f7cf006ff20 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 1 -- 10.20.193.18:0/131130490 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0037440 con 0x7f7cf006ff20 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf544d640 10 monclient: dump: 2026-02-24T18:36:09.214 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 4 set_mon_vals no callback set 2026-02-24T18:36:09.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 10 monclient: shutdown 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 -- 10.20.193.18:0/131130490 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf006ff20 msgr2=0x7f7cf0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 --2- 10.20.193.18:0/131130490 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf006ff20 0x7f7cf0070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009970 tx=0x7f7ce002eef0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 -- 10.20.193.18:0/131130490 shutdown_connections 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 --2- 10.20.193.18:0/131130490 >> v2:10.20.193.32:3300/0 conn(0x7f7cf015a5e0 0x7f7cf014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 --2- 10.20.193.18:0/131130490 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf006ff20 0x7f7cf0070300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 -- 10.20.193.18:0/131130490 >> 10.20.193.18:0/131130490 conn(0x7f7cf00cd780 msgr2=0x7f7cf00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 -- 10.20.193.18:0/131130490 shutdown_connections 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.201+0000 7f7cf86da640 1 -- 10.20.193.18:0/131130490 wait complete. 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 10 monclient: build_initial_monmap 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 10 monclient: monmap: 2026-02-24T18:36:09.216 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 librados: starting msgr at 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 librados: starting objecter 2026-02-24T18:36:09.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 Processor -- start 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 -- start start 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 librados: setting wanted keys 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 1 librados: calling monclient init 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.202+0000 7f7cf86da640 10 monclient: init 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7cf006ff20 0x7f7cf01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-c con 0x7f7cf006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-a con 0x7f7cf00c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7cf015a5e0 0x7f7cf0072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): picked mon.noname-b con 0x7f7cf015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0150350 con 0x7f7cf00c99a0 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf00c8a10 con 0x7f7cf015a5e0 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf01504d0 con 0x7f7cf006ff20 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf86da640 10 monclient(hunting): authenticate will time out at 1512.885420s 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf6c50640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7cf015a5e0 0x7f7cf0072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47150/0 (socket says 10.20.193.18:47150) 2026-02-24T18:36:09.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 1 -- 10.20.193.18:0/1315141858 learned_addr learned my addr 10.20.193.18:0/1315141858 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 10 monclient(hunting): get_auth_request con 0x7f7cf00c99a0 auth_method 0 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf5c4e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.203+0000 7f7cf644f640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7cf006ff20 0x7f7cf01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): get_auth_request con 0x7f7cf015a5e0 auth_method 0 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): get_auth_request con 0x7f7cf006ff20 auth_method 0 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:09.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf5c4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf5c4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf5c4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf6c50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.204+0000 7f7cf644f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 10 monclient(hunting): handle_auth_done global_id 4704 payload 824 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 -- 10.20.193.18:0/1315141858 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7cf006ff20 msgr2=0x7f7cf01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7cf006ff20 0x7f7cf01f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 -- 10.20.193.18:0/1315141858 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7cf015a5e0 msgr2=0x7f7cf0072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7cf015a5e0 0x7f7cf0072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 10 monclient: _finish_hunting 0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 monclient: found mon.noname-a 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.205+0000 7f7cf5c4e640 1 -- 10.20.193.18:0/1315141858 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf00c10d0 con 0x7f7cf00c99a0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.206+0000 7f7cf5c4e640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c580 tx=0x7f7ce400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.206+0000 7f7cd6ffd640 10 client.?.objecter ms_handle_connect 0x7f7cf00c99a0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.206+0000 7f7cd6ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4017070 con 0x7f7cf00c99a0 2026-02-24T18:36:09.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: dump: 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce40052d0 con 0x7f7cf00c99a0 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40036b0 con 0x7f7cf00c99a0 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cd6ffd640 10 monclient: dump: 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.224 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf4c4c640 4 set_mon_vals no callback set 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 5 monclient: authenticate success, global_id 4704 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 10 monclient: _renew_subs 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf01f3c70 con 0x7f7cf00c99a0 2026-02-24T18:36:09.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 10 client.4704.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 10 monclient: _renew_subs 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf01f4220 con 0x7f7cf00c99a0 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.208+0000 7f7cf86da640 1 librados: init done 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.209+0000 7f7cf86da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.209+0000 7f7cf86da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.209+0000 7f7cf86da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.209+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf0150860 con 0x7f7cf00c99a0 2026-02-24T18:36:09.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.315+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7ce4003850 con 0x7f7cf00c99a0 2026-02-24T18:36:09.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.316+0000 7f7cd6ffd640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 0x7f7cc007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:09.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.316+0000 7f7cf644f640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 0x7f7cc007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:09.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.316+0000 7f7cf644f640 10 monclient: get_auth_request con 0x7f7cc0078e00 auth_method 0 2026-02-24T18:36:09.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.317+0000 7f7cd6ffd640 10 client.4704.objecter ms_handle_connect 0x7f7cc0078e00 2026-02-24T18:36:09.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.317+0000 7f7cf644f640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 0x7f7cc007b2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00c5350 tx=0x7f7ce0039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:09.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7ce405d470 con 0x7f7cf00c99a0 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 10 client.4704.objecter ms_dispatch2 0x7f7cf0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 3 client.4704.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 3 client.4704.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 20 client.4704.objecter dump_active .. 0 homeless 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7ce4021020 con 0x7f7cf00c99a0 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:09.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.324+0000 7f7cd6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:09.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.424+0000 7f7cf86da640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:09.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.424+0000 7f7cf86da640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:09.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.424+0000 7f7cf86da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:09.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.424+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7cf014f510 con 0x7f7cf00c99a0 2026-02-24T18:36:09.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.427+0000 7f7cd6ffd640 1 -- 10.20.193.18:0/1315141858 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7ce4061430 con 0x7f7cf00c99a0 2026-02-24T18:36:09.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.427+0000 7f7cd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:09.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.427+0000 7f7cd6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:09.430 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:09.431 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:09.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 10 librados: watch_flush enter 2026-02-24T18:36:09.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 10 librados: watch_flush exit 2026-02-24T18:36:09.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 20 client.4704.objecter shutdown clearing up homeless session... 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 10 client.4704.objecter successfully canceled tick 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 msgr2=0x7f7cc007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 0x7f7cc007b2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00c5350 tx=0x7f7ce0039040 comp rx=0 tx=0).stop 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 10 monclient: shutdown 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 msgr2=0x7f7cf01f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c580 tx=0x7f7ce400ca50 comp rx=0 tx=0).stop 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 shutdown_connections 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cc0078e00 0x7f7cc007b2c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7cf015a5e0 0x7f7cf0072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7cf00c99a0 0x7f7cf01f3730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 --2- 10.20.193.18:0/1315141858 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7cf006ff20 0x7f7cf01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:09.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 >> 10.20.193.18:0/1315141858 conn(0x7f7cf00cd780 msgr2=0x7f7cf014e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:09.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 shutdown_connections 2026-02-24T18:36:09.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.428+0000 7f7cf86da640 1 -- 10.20.193.18:0/1315141858 wait complete. 2026-02-24T18:36:09.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:09.429+0000 7f7cf86da640 1 librados: shutdown 2026-02-24T18:36:12.441 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:12.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.511+0000 7fd8cef53640 10 monclient: get_monmap_and_config 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.511+0000 7fd8cef53640 10 monclient: build_initial_monmap 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.511+0000 7fd8cef53640 10 monclient: monmap: 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:12.512587+0000 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:12.512587+0000 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:12.514 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 Processor -- start 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 -- start start 2026-02-24T18:36:12.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient: init 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient: _add_conns ranks=[3,5,0,2,4,1] 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd8c806ff20 0x7fd8c8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-a con 0x7fd8c806ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd8c80c99a0 0x7fd8c8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-c con 0x7fd8c80c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:12.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8c815a410 addr v1:10.20.193.18:6789/0 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c815a860 con 0x7fd8c815a410 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c8159a40 con 0x7fd8c806ff20 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c80c7e90 con 0x7fd8c80c99a0 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.512+0000 7fd8cef53640 10 monclient(hunting): authenticate will time out at 1516.193708s 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd8c806ff20 0x7fd8c8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd8c806ff20 0x7fd8c8070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47162/0 (socket says 10.20.193.18:47162) 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 1 -- 10.20.193.18:0/769944044 learned_addr learned my addr 10.20.193.18:0/769944044 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:12.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): get_auth_request con 0x7fd8c806ff20 auth_method 0 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.513+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_done global_id 4707 payload 274 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 1 -- 10.20.193.18:0/769944044 >> v2:10.20.193.110:3300/0 conn(0x7fd8c80c99a0 msgr2=0x7fd8c8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 1 --2- 10.20.193.18:0/769944044 >> v2:10.20.193.110:3300/0 conn(0x7fd8c80c99a0 0x7fd8c8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 1 -- 10.20.193.18:0/769944044 >> v1:10.20.193.18:6789/0 conn(0x7fd8c815a410 legacy=0x7fd8c814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 10 monclient: _finish_hunting 0 2026-02-24T18:36:12.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 1 monclient: found mon.noname-a 2026-02-24T18:36:12.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:12.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.514+0000 7fd8cccc8640 1 -- 10.20.193.18:0/769944044 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c80c8170 con 0x7fd8c806ff20 2026-02-24T18:36:12.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.515+0000 7fd8cccc8640 1 --2- 10.20.193.18:0/769944044 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c806ff20 0x7fd8c8070300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009970 tx=0x7fd8b802eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e02c389f6de811a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:12.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 1 -- 10.20.193.18:0/769944044 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b803c070 con 0x7fd8c806ff20 2026-02-24T18:36:12.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:12.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:12.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: dump: 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: _finish_auth 0 2026-02-24T18:36:12.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 1 -- 10.20.193.18:0/769944044 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b802fb20 con 0x7fd8c806ff20 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8cef53640 5 monclient: authenticate success, global_id 4707 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 1 -- 10.20.193.18:0/769944044 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8037440 con 0x7fd8c806ff20 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8cef53640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8cef53640 4 set_mon_vals no callback set 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.517+0000 7fd8bf7fe640 10 monclient: dump: 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:12.529 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 10 monclient: shutdown 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 -- 10.20.193.18:0/769944044 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c806ff20 msgr2=0x7fd8c8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:12.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 --2- 10.20.193.18:0/769944044 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c806ff20 0x7fd8c8070300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009970 tx=0x7fd8b802eef0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 -- 10.20.193.18:0/769944044 shutdown_connections 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 --2- 10.20.193.18:0/769944044 >> v2:10.20.193.110:3300/0 conn(0x7fd8c80c99a0 0x7fd8c8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 --2- 10.20.193.18:0/769944044 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c806ff20 0x7fd8c8070300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 -- 10.20.193.18:0/769944044 >> 10.20.193.18:0/769944044 conn(0x7fd8c80cd780 msgr2=0x7fd8c80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 -- 10.20.193.18:0/769944044 shutdown_connections 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 -- 10.20.193.18:0/769944044 wait complete. 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 10 monclient: build_initial_monmap 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 10 monclient: monmap: 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:12.531 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 librados: starting msgr at 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 librados: starting objecter 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.518+0000 7fd8cef53640 1 Processor -- start 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 -- start start 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 librados: setting wanted keys 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 librados: calling monclient init 2026-02-24T18:36:12.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient: init 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd8c806ff20 0x7fd8c81f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-b con 0x7fd8c806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-a con 0x7fd8c80c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd8c815a410 0x7fd8c81f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): picked mon.noname-c con 0x7fd8c815a410 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c80c8a10 con 0x7fd8c80c99a0 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:12.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c80c8730 con 0x7fd8c806ff20 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8150520 con 0x7fd8c815a410 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cef53640 10 monclient(hunting): authenticate will time out at 1516.200416s 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47168/0 (socket says 10.20.193.18:47168) 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 1 -- 10.20.193.18:0/1467078197 learned_addr learned my addr 10.20.193.18:0/1467078197 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): get_auth_request con 0x7fd8c80c99a0 auth_method 0 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:12.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8cd4c9640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd8c815a410 0x7fd8c81f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.519+0000 7fd8bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd8c806ff20 0x7fd8c81f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): get_auth_request con 0x7fd8c815a410 auth_method 0 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): get_auth_request con 0x7fd8c806ff20 auth_method 0 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:12.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cd4c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.520+0000 7fd8cccc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 10 monclient(hunting): handle_auth_done global_id 4710 payload 824 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 -- 10.20.193.18:0/1467078197 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd8c815a410 msgr2=0x7fd8c81f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd8c815a410 0x7fd8c81f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 -- 10.20.193.18:0/1467078197 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd8c806ff20 msgr2=0x7fd8c81f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd8c806ff20 0x7fd8c81f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 10 monclient: _finish_hunting 0 2026-02-24T18:36:12.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 monclient: found mon.noname-a 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.521+0000 7fd8bffff640 1 -- 10.20.193.18:0/1467078197 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c80cca40 con 0x7fd8c80c99a0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.522+0000 7fd8bffff640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b780 tx=0x7fd8b000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.522+0000 7fd8bd7fa640 10 client.?.objecter ms_handle_connect 0x7fd8c80c99a0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.522+0000 7fd8bd7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0004480 con 0x7fd8c80c99a0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 10 monclient: dump: 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:12.537 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 10 monclient: _finish_auth 0 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:12.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b0010070 con 0x7fd8c80c99a0 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.523+0000 7fd8cef53640 5 monclient: authenticate success, global_id 4710 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8bd7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0002920 con 0x7fd8c80c99a0 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _renew_subs 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8beffd640 4 set_mon_vals no callback set 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81f5070 con 0x7fd8c80c99a0 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8bd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8bd7fa640 10 monclient: dump: 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:12.539 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 client.4710.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _renew_subs 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81f5630 con 0x7fd8c80c99a0 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 1 librados: init done 2026-02-24T18:36:12.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:12.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:12.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.524+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c8070300 con 0x7fd8c80c99a0 2026-02-24T18:36:12.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.629+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd8b001d030 con 0x7fd8c80c99a0 2026-02-24T18:36:12.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.630+0000 7fd8bd7fa640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 0x7fd89807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:12.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.630+0000 7fd8cccc8640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 0x7fd89807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:12.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.630+0000 7fd8cccc8640 10 monclient: get_auth_request con 0x7fd898078e50 auth_method 0 2026-02-24T18:36:12.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.630+0000 7fd8cccc8640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 0x7fd89807b310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80c5380 tx=0x7fd8b8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:12.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.630+0000 7fd8bd7fa640 10 client.4710.objecter ms_handle_connect 0x7fd898078e50 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd8b005d280 con 0x7fd8c80c99a0 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 10 client.4710.objecter ms_dispatch2 0x7fd8c8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 3 client.4710.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 3 client.4710.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 20 client.4710.objecter dump_active .. 0 homeless 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd8b0024080 con 0x7fd8c80c99a0 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:12.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.639+0000 7fd8bd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:12.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.741+0000 7fd8cef53640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:12.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.741+0000 7fd8cef53640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:12.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.741+0000 7fd8cef53640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:12.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.741+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd8c81f61e0 con 0x7fd8c80c99a0 2026-02-24T18:36:12.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.744+0000 7fd8bd7fa640 1 -- 10.20.193.18:0/1467078197 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd8b0061240 con 0x7fd8c80c99a0 2026-02-24T18:36:12.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.744+0000 7fd8bd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:12.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.744+0000 7fd8bd7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:12.746 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:12.746 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 10 librados: watch_flush enter 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 10 librados: watch_flush exit 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 20 client.4710.objecter shutdown clearing up homeless session... 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 10 client.4710.objecter successfully canceled tick 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 msgr2=0x7fd89807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 0x7fd89807b310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80c5380 tx=0x7fd8b8033000 comp rx=0 tx=0).stop 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 10 monclient: shutdown 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 msgr2=0x7fd8c81f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:12.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b780 tx=0x7fd8b000bc50 comp rx=0 tx=0).stop 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 shutdown_connections 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd898078e50 0x7fd89807b310 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd8c815a410 0x7fd8c81f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd8c80c99a0 0x7fd8c81f7170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 --2- 10.20.193.18:0/1467078197 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd8c806ff20 0x7fd8c81f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 >> 10.20.193.18:0/1467078197 conn(0x7fd8c80cd780 msgr2=0x7fd8c8072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 shutdown_connections 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 -- 10.20.193.18:0/1467078197 wait complete. 2026-02-24T18:36:12.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:12.745+0000 7fd8cef53640 1 librados: shutdown 2026-02-24T18:36:15.758 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:15.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.826+0000 7f1176a6d640 10 monclient: get_monmap_and_config 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.826+0000 7f1176a6d640 10 monclient: build_initial_monmap 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.826+0000 7f1176a6d640 10 monclient: monmap: 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:15.827761+0000 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:15.827761+0000 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.829 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 Processor -- start 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 -- start start 2026-02-24T18:36:15.830 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient: init 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient: _add_conns ranks=[1,2,5,3,4,0] 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f117006ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11700c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f117015a5e0 0x7f1170070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-c con 0x7f117015a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700c7e90 con 0x7f117006ff20 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700c8170 con 0x7f11700c99a0 2026-02-24T18:36:15.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1170159a40 con 0x7f117015a5e0 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.827+0000 7f1176a6d640 10 monclient(hunting): authenticate will time out at 1519.508888s 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f116ffff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f117006ff20 0x7f1170159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:39612/0 (socket says 10.20.193.18:39612) 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f116ffff640 1 -- 10.20.193.18:0/1058970934 learned_addr learned my addr 10.20.193.18:0/1058970934 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f1174fe3640 1 --2- 10.20.193.18:0/1058970934 >> v2:10.20.193.110:3300/0 conn(0x7f117015a5e0 0x7f1170070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f1174fe3640 10 monclient(hunting): get_auth_request con 0x7f117015a5e0 auth_method 0 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f1174fe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f1174fe3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.828+0000 7f1174fe3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:15.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 1 -- 10.20.193.18:0/1058970934 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377050364 0 0) 0x7f11700c7e90 con 0x7f117006ff20 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 10 monclient(hunting): _negotiate 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 10 monclient(hunting): my global_id is 4696 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.829+0000 7f116effd640 1 -- 10.20.193.18:0/1058970934 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1154003620 con 0x7f117006ff20 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 10 monclient(hunting): handle_auth_done global_id 4658 payload 274 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 1 -- 10.20.193.18:0/1058970934 >> v1:10.20.193.110:6789/0 conn(0x7f11700c99a0 legacy=0x7f117014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 1 -- 10.20.193.18:0/1058970934 >> v1:10.20.193.32:6789/0 conn(0x7f117006ff20 legacy=0x7f1170159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 10 monclient: _finish_hunting 0 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 1 monclient: found mon.noname-c 2026-02-24T18:36:15.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:15.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:36:15.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.830+0000 7f1174fe3640 1 -- 10.20.193.18:0/1058970934 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700c8450 con 0x7f117015a5e0 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.833+0000 7f1174fe3640 1 --2- 10.20.193.18:0/1058970934 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1164009dc0 tx=0x7f116402f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71f935c178b759f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 1 -- 10.20.193.18:0/1058970934 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403c070 con 0x7f117015a5e0 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: dump: 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.841 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: _finish_auth 0 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 1 -- 10.20.193.18:0/1058970934 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164037440 con 0x7f117015a5e0 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 5 monclient: authenticate success, global_id 4658 2026-02-24T18:36:15.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 1 -- 10.20.193.18:0/1058970934 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037810 con 0x7f117015a5e0 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 4 set_mon_vals no callback set 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f116effd640 10 monclient: dump: 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.843 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 10 monclient: shutdown 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 -- 10.20.193.18:0/1058970934 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 msgr2=0x7f1170070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 --2- 10.20.193.18:0/1058970934 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1164009dc0 tx=0x7f116402f280 comp rx=0 tx=0).stop 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 -- 10.20.193.18:0/1058970934 shutdown_connections 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 --2- 10.20.193.18:0/1058970934 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170070300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:15.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 -- 10.20.193.18:0/1058970934 >> 10.20.193.18:0/1058970934 conn(0x7f11700cd780 msgr2=0x7f11700cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 -- 10.20.193.18:0/1058970934 shutdown_connections 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.835+0000 7f1176a6d640 1 -- 10.20.193.18:0/1058970934 wait complete. 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient: build_initial_monmap 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient: monmap: 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:15.845 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 librados: starting msgr at 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 librados: starting objecter 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 Processor -- start 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 -- start start 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 librados: setting wanted keys 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 librados: calling monclient init 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient: init 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 0x7f11700ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:15.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-a con 0x7f117006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 0x7f1170158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-b con 0x7f11700c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): picked mon.noname-c con 0x7f117015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1170150520 con 0x7f117006ff20 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11700c8a10 con 0x7f11700c99a0 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11701506a0 con 0x7f117015a5e0 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.836+0000 7f1176a6d640 10 monclient(hunting): authenticate will time out at 1519.517972s 2026-02-24T18:36:15.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 0x7f11700ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 0x7f1170158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 0x7f11700ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47184/0 (socket says 10.20.193.18:47184) 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 1 -- 10.20.193.18:0/3201915330 learned_addr learned my addr 10.20.193.18:0/3201915330 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): get_auth_request con 0x7f117006ff20 auth_method 0 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): get_auth_request con 0x7f117015a5e0 auth_method 0 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:15.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): get_auth_request con 0x7f11700c99a0 auth_method 0 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f1174fe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:15.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.837+0000 7f116ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 10 monclient(hunting): handle_auth_done global_id 4699 payload 824 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 -- 10.20.193.18:0/3201915330 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 msgr2=0x7f1170158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 -- 10.20.193.18:0/3201915330 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 msgr2=0x7f11700ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 0x7f11700ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 monclient: found mon.noname-b 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:15.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.839+0000 7f116f7fe640 1 -- 10.20.193.18:0/3201915330 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701552a0 con 0x7f11700c99a0 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.840+0000 7f116f7fe640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 0x7f1170158250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f116000c480 tx=0x7f116000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.840+0000 7f116cff9640 10 client.?.objecter ms_handle_connect 0x7f11700c99a0 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.840+0000 7f116cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160017070 con 0x7f11700c99a0 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: dump: 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.853 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: _finish_auth 0 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11600052d0 con 0x7f11700c99a0 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:15.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600036f0 con 0x7f11700c99a0 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116cff9640 10 monclient: dump: 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:15.855 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 5 monclient: authenticate success, global_id 4699 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f116e7fc640 4 set_mon_vals no callback set 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _renew_subs 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1170155580 con 0x7f11700c99a0 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 client.4699.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _renew_subs 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1170157a50 con 0x7f11700c99a0 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 1 librados: init done 2026-02-24T18:36:15.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:15.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:15.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:15.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.842+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170157840 con 0x7f11700c99a0 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.853+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1160003890 con 0x7f11700c99a0 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.854+0000 7f116cff9640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 0x7f114007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.854+0000 7f116ffff640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 0x7f114007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.854+0000 7f116ffff640 10 monclient: get_auth_request con 0x7f1140078e30 auth_method 0 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.854+0000 7f116ffff640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 0x7f114007b2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f115c005980 tx=0x7f115c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.854+0000 7f116cff9640 10 client.4699.objecter ms_handle_connect 0x7f1140078e30 2026-02-24T18:36:15.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.855+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f116005d320 con 0x7f11700c99a0 2026-02-24T18:36:15.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.855+0000 7f116cff9640 10 client.4699.objecter ms_dispatch2 0x7f1170000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:15.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.855+0000 7f116cff9640 3 client.4699.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:15.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.855+0000 7f116cff9640 3 client.4699.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:15.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.855+0000 7f116cff9640 20 client.4699.objecter dump_active .. 0 homeless 2026-02-24T18:36:15.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.954+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1160021020 con 0x7f11700c99a0 2026-02-24T18:36:15.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.954+0000 7f116cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:15.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:15.954+0000 7f116cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:16.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.057+0000 7f1176a6d640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:16.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.057+0000 7f1176a6d640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:16.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.057+0000 7f1176a6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:16.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.057+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1170070300 con 0x7f11700c99a0 2026-02-24T18:36:16.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.059+0000 7f116cff9640 1 -- 10.20.193.18:0/3201915330 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f11600612e0 con 0x7f11700c99a0 2026-02-24T18:36:16.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.059+0000 7f116cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:16.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.059+0000 7f116cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:16.061 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:16.061 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 10 librados: watch_flush enter 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 10 librados: watch_flush exit 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 20 client.4699.objecter shutdown clearing up homeless session... 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 10 client.4699.objecter successfully canceled tick 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 msgr2=0x7f114007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 0x7f114007b2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f115c005980 tx=0x7f115c009040 comp rx=0 tx=0).stop 2026-02-24T18:36:16.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 10 monclient: shutdown 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 msgr2=0x7f1170158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 0x7f1170158250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f116000c480 tx=0x7f116000c950 comp rx=0 tx=0).stop 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 shutdown_connections 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1140078e30 0x7f114007b2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f11700c99a0 0x7f1170158250 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f117015a5e0 0x7f1170158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 --2- 10.20.193.18:0/3201915330 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f117006ff20 0x7f11700ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 >> 10.20.193.18:0/3201915330 conn(0x7f11700cd780 msgr2=0x7f117014e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 shutdown_connections 2026-02-24T18:36:16.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 -- 10.20.193.18:0/3201915330 wait complete. 2026-02-24T18:36:16.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:16.060+0000 7f1176a6d640 1 librados: shutdown 2026-02-24T18:36:19.072 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:19.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 10 monclient: get_monmap_and_config 2026-02-24T18:36:19.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 10 monclient: build_initial_monmap 2026-02-24T18:36:19.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 10 monclient: monmap: 2026-02-24T18:36:19.143 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:19.142237+0000 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:19.142237+0000 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:19.144 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 1 Processor -- start 2026-02-24T18:36:19.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 1 -- start start 2026-02-24T18:36:19.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.141+0000 7fd34b098640 10 monclient: init 2026-02-24T18:36:19.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:19.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient: _add_conns ranks=[3,4,2,1,5,0] 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd34406ff20 0x7fd344070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-a con 0x7fd34406ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fd3440c99a0 0x7fd344159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-b con 0x7fd3440c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd34415a410 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34415a860 con 0x7fd34415a410 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd344159a40 con 0x7fd34406ff20 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd3440c7e90 con 0x7fd3440c99a0 2026-02-24T18:36:19.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd34b098640 10 monclient(hunting): authenticate will time out at 1522.823381s 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd34406ff20 0x7fd344070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd33bfff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fd3440c99a0 0x7fd344159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd34406ff20 0x7fd344070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53332/0 (socket says 10.20.193.18:53332) 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 1 -- 10.20.193.18:0/1543895127 learned_addr learned my addr 10.20.193.18:0/1543895127 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 10 monclient(hunting): get_auth_request con 0x7fd34406ff20 auth_method 0 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.142+0000 7fd348e0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): get_auth_request con 0x7fd3440c99a0 auth_method 0 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:19.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.143+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 10 monclient(hunting): handle_auth_done global_id 4716 payload 274 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 1 -- 10.20.193.18:0/1543895127 >> v2:10.20.193.32:3300/0 conn(0x7fd3440c99a0 msgr2=0x7fd344159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 1 --2- 10.20.193.18:0/1543895127 >> v2:10.20.193.32:3300/0 conn(0x7fd3440c99a0 0x7fd344159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 1 -- 10.20.193.18:0/1543895127 >> v1:10.20.193.110:6789/0 conn(0x7fd34415a410 legacy=0x7fd34414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 10 monclient: _finish_hunting 0 2026-02-24T18:36:19.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 1 monclient: found mon.noname-a 2026-02-24T18:36:19.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:19.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.144+0000 7fd348e0d640 1 -- 10.20.193.18:0/1543895127 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3440c8170 con 0x7fd34406ff20 2026-02-24T18:36:19.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.145+0000 7fd348e0d640 1 --2- 10.20.193.18:0/1543895127 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd344070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd334009970 tx=0x7fd33402eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8370bf76b36363 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:19.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 1 -- 10.20.193.18:0/1543895127 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33403c070 con 0x7fd34406ff20 2026-02-24T18:36:19.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: dump: 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:19.158 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 1 -- 10.20.193.18:0/1543895127 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33402fbc0 con 0x7fd34406ff20 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd34b098640 5 monclient: authenticate success, global_id 4716 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd34b098640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 1 -- 10.20.193.18:0/1543895127 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334037440 con 0x7fd34406ff20 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd34b098640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd34b098640 4 set_mon_vals no callback set 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:19.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.146+0000 7fd33b7fe640 10 monclient: dump: 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 10 monclient: shutdown 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- 10.20.193.18:0/1543895127 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 msgr2=0x7fd344070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 --2- 10.20.193.18:0/1543895127 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd344070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd334009970 tx=0x7fd33402eef0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- 10.20.193.18:0/1543895127 shutdown_connections 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 --2- 10.20.193.18:0/1543895127 >> v2:10.20.193.32:3300/0 conn(0x7fd3440c99a0 0x7fd344159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 --2- 10.20.193.18:0/1543895127 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd344070300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- 10.20.193.18:0/1543895127 >> 10.20.193.18:0/1543895127 conn(0x7fd3440cd780 msgr2=0x7fd3440cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- 10.20.193.18:0/1543895127 shutdown_connections 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- 10.20.193.18:0/1543895127 wait complete. 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 10 monclient: build_initial_monmap 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 10 monclient: monmap: 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:19.161 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 librados: starting msgr at 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 librados: starting objecter 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 Processor -- start 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 -- start start 2026-02-24T18:36:19.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 librados: setting wanted keys 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 1 librados: calling monclient init 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.147+0000 7fd34b098640 10 monclient: init 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-a con 0x7fd34406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3440c99a0 0x7fd3441f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-c con 0x7fd3440c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd34415a410 0x7fd3441f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): picked mon.noname-b con 0x7fd34415a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440c8a10 con 0x7fd34406ff20 2026-02-24T18:36:19.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440c8730 con 0x7fd34415a410 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd344150520 con 0x7fd3440c99a0 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34b098640 10 monclient(hunting): authenticate will time out at 1522.829221s 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53342/0 (socket says 10.20.193.18:53342) 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 1 -- 10.20.193.18:0/2873822272 learned_addr learned my addr 10.20.193.18:0/2873822272 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): get_auth_request con 0x7fd34406ff20 auth_method 0 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:19.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34960e640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd34415a410 0x7fd3441f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd33bfff640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3440c99a0 0x7fd3441f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd348e0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34960e640 10 monclient(hunting): get_auth_request con 0x7fd34415a410 auth_method 0 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34960e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34960e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd34960e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd33bfff640 10 monclient(hunting): get_auth_request con 0x7fd3440c99a0 auth_method 0 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd33bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd33bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:19.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.148+0000 7fd33bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd34960e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd34960e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd34960e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.149+0000 7fd33bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 10 monclient(hunting): handle_auth_done global_id 4719 payload 824 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 -- 10.20.193.18:0/2873822272 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3440c99a0 msgr2=0x7fd3441f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3440c99a0 0x7fd3441f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 -- 10.20.193.18:0/2873822272 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd34415a410 msgr2=0x7fd3441f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd34415a410 0x7fd3441f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 10 monclient: _finish_hunting 0 2026-02-24T18:36:19.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 monclient: found mon.noname-a 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.150+0000 7fd348e0d640 1 -- 10.20.193.18:0/2873822272 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3441f45a0 con 0x7fd34406ff20 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.151+0000 7fd348e0d640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd334009aa0 tx=0x7fd3340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.151+0000 7fd3397fa640 10 client.?.objecter ms_handle_connect 0x7fd34406ff20 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.151+0000 7fd3397fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33403c040 con 0x7fd34406ff20 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: dump: 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:19.167 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: _finish_auth 0 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: _check_auth_tickets 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd334042410 con 0x7fd34406ff20 2026-02-24T18:36:19.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334042800 con 0x7fd34406ff20 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd33affd640 4 set_mon_vals no callback set 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd3397fa640 10 monclient: dump: 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:19.169 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 5 monclient: authenticate success, global_id 4719 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _renew_subs 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3440c2510 con 0x7fd34406ff20 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 client.4719.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _renew_subs 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3440c2ad0 con 0x7fd34406ff20 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 1 librados: init done 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:19.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:19.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.153+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd344070300 con 0x7fd34406ff20 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.165+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd3340429a0 con 0x7fd34406ff20 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.165+0000 7fd3397fa640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 0x7fd31407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.165+0000 7fd33bfff640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 0x7fd31407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.165+0000 7fd33bfff640 10 monclient: get_auth_request con 0x7fd314078e50 auth_method 0 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.166+0000 7fd33bfff640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 0x7fd31407b310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd344157af0 tx=0x7fd32c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:19.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.166+0000 7fd3397fa640 10 client.4719.objecter ms_handle_connect 0x7fd314078e50 2026-02-24T18:36:19.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.167+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd334083630 con 0x7fd34406ff20 2026-02-24T18:36:19.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.167+0000 7fd3397fa640 10 client.4719.objecter ms_dispatch2 0x7fd344000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:19.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.167+0000 7fd3397fa640 3 client.4719.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:19.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.167+0000 7fd3397fa640 3 client.4719.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:19.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.167+0000 7fd3397fa640 20 client.4719.objecter dump_active .. 0 homeless 2026-02-24T18:36:19.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.265+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd3340875f0 con 0x7fd34406ff20 2026-02-24T18:36:19.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.265+0000 7fd3397fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:19.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.265+0000 7fd3397fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:19.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.364+0000 7fd34b098640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:19.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.364+0000 7fd34b098640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:19.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.364+0000 7fd34b098640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:19.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.364+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd3441573f0 con 0x7fd34406ff20 2026-02-24T18:36:19.377 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:57.322 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.378+0000 7fd3397fa640 1 -- 10.20.193.18:0/2873822272 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd33408b8f0 con 0x7fd34406ff20 2026-02-24T18:36:19.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.379+0000 7fd3397fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:19.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.379+0000 7fd3397fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:19.381 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:19.381 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:19.381 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:57.327 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 10 librados: watch_flush enter 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 10 librados: watch_flush exit 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 20 client.4719.objecter shutdown clearing up homeless session... 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 10 client.4719.objecter successfully canceled tick 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 msgr2=0x7fd31407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 0x7fd31407b310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd344157af0 tx=0x7fd32c009290 comp rx=0 tx=0).stop 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 10 monclient: shutdown 2026-02-24T18:36:19.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 msgr2=0x7fd34414cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd334009aa0 tx=0x7fd3340047c0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 shutdown_connections 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd314078e50 0x7fd31407b310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd34415a410 0x7fd3441f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3440c99a0 0x7fd3441f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 --2- 10.20.193.18:0/2873822272 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd34406ff20 0x7fd34414cca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 >> 10.20.193.18:0/2873822272 conn(0x7fd3440cd780 msgr2=0x7fd344157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 shutdown_connections 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 -- 10.20.193.18:0/2873822272 wait complete. 2026-02-24T18:36:19.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:19.380+0000 7fd34b098640 1 librados: shutdown 2026-02-24T18:36:19.392 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:02:57.336 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.406 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:57.351 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.407 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:57.352 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.408 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:57.316 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.409 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:57.317 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.415 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:02:57.360 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:19.416 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:02:57.325 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:22.393 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.461+0000 7ff48bbe3640 10 monclient: get_monmap_and_config 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.461+0000 7ff48bbe3640 10 monclient: build_initial_monmap 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.461+0000 7ff48bbe3640 10 monclient: monmap: 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:22.462747+0000 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:22.462747+0000 2026-02-24T18:36:22.464 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 Processor -- start 2026-02-24T18:36:22.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 -- start start 2026-02-24T18:36:22.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient: init 2026-02-24T18:36:22.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient: _add_conns ranks=[2,3,4,1,0,5] 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff48406ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 0x7ff484070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-a con 0x7ff4840c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff48415a5e0 0x7ff48414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-b con 0x7ff48415a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4840c7e90 con 0x7ff48406ff20 2026-02-24T18:36:22.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff484159a40 con 0x7ff4840c99a0 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff4840c8170 con 0x7ff48415a5e0 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.462+0000 7ff48bbe3640 10 monclient(hunting): authenticate will time out at 1526.143920s 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 0x7ff484070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 0x7ff484070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53352/0 (socket says 10.20.193.18:53352) 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 1 -- 10.20.193.18:0/3304665090 learned_addr learned my addr 10.20.193.18:0/3304665090 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): get_auth_request con 0x7ff4840c99a0 auth_method 0 2026-02-24T18:36:22.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff48a159640 1 --2- 10.20.193.18:0/3304665090 >> v2:10.20.193.32:3300/0 conn(0x7ff48415a5e0 0x7ff48414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff48a159640 10 monclient(hunting): get_auth_request con 0x7ff48415a5e0 auth_method 0 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff48a159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff48a159640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff48a159640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.463+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:22.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.464+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.464+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.464+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.464+0000 7ff48a159640 10 monclient(hunting): handle_auth_done global_id 4708 payload 274 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 1 -- 10.20.193.18:0/3304665090 >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 msgr2=0x7ff484070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 1 --2- 10.20.193.18:0/3304665090 >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 0x7ff484070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 1 -- 10.20.193.18:0/3304665090 >> v1:10.20.193.110:6789/0 conn(0x7ff48406ff20 legacy=0x7ff484159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 10 monclient: _finish_hunting 0 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 1 monclient: found mon.noname-b 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:22.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.465+0000 7ff48a159640 1 -- 10.20.193.18:0/3304665090 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4840c8450 con 0x7ff48415a5e0 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.466+0000 7ff48a159640 1 --2- 10.20.193.18:0/3304665090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff48415a5e0 0x7ff48414fe10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff48002f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bffe559ea56ae92 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 1 -- 10.20.193.18:0/3304665090 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48003c070 con 0x7ff48415a5e0 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: dump: 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.481 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: _finish_auth 0 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: _check_auth_tickets 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff48bbe3640 5 monclient: authenticate success, global_id 4708 2026-02-24T18:36:22.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff48bbe3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 1 -- 10.20.193.18:0/3304665090 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480037440 con 0x7ff48415a5e0 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 1 -- 10.20.193.18:0/3304665090 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037810 con 0x7ff48415a5e0 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff488956640 10 monclient: dump: 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.483 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff48bbe3640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.467+0000 7ff48bbe3640 4 set_mon_vals no callback set 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 10 monclient: shutdown 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:22.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3304665090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff48415a5e0 msgr2=0x7ff48414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3304665090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff48415a5e0 0x7ff48414fe10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff480009d90 tx=0x7ff48002f1e0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3304665090 shutdown_connections 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3304665090 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff48415a5e0 0x7ff48414fe10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3304665090 >> v2:10.20.193.18:3300/0 conn(0x7ff4840c99a0 0x7ff484070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3304665090 >> 10.20.193.18:0/3304665090 conn(0x7ff4840cd780 msgr2=0x7ff4840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3304665090 shutdown_connections 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3304665090 wait complete. 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 10 monclient: build_initial_monmap 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 10 monclient: monmap: 2026-02-24T18:36:22.485 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 librados: starting msgr at 2026-02-24T18:36:22.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 librados: starting objecter 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.468+0000 7ff48bbe3640 1 Processor -- start 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 -- start start 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 librados: setting wanted keys 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 librados: calling monclient init 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient: init 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 0x7ff4841f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-a con 0x7ff48406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 0x7ff48406e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-b con 0x7ff4840c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff48415a5e0 0x7ff4840c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): picked mon.noname-c con 0x7ff48415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff484150350 con 0x7ff48406ff20 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840c8a10 con 0x7ff4840c99a0 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841504d0 con 0x7ff48415a5e0 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48bbe3640 10 monclient(hunting): authenticate will time out at 1526.150423s 2026-02-24T18:36:22.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 0x7ff4841f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489157640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 0x7ff48406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 0x7ff4841f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53366/0 (socket says 10.20.193.18:53366) 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 1 -- 10.20.193.18:0/3789170335 learned_addr learned my addr 10.20.193.18:0/3789170335 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 10 monclient(hunting): get_auth_request con 0x7ff48406ff20 auth_method 0 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489958640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489157640 10 monclient(hunting): get_auth_request con 0x7ff4840c99a0 auth_method 0 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489157640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:22.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff489157640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.469+0000 7ff48a159640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff48415a5e0 0x7ff4840c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489958640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489958640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489958640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff489157640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): get_auth_request con 0x7ff48415a5e0 auth_method 0 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:22.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.470+0000 7ff48a159640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 10 monclient(hunting): handle_auth_done global_id 4711 payload 824 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 -- 10.20.193.18:0/3789170335 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff48415a5e0 msgr2=0x7ff4840c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff48415a5e0 0x7ff4840c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 -- 10.20.193.18:0/3789170335 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 msgr2=0x7ff4841f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 0x7ff4841f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 10 monclient: _finish_hunting 0 2026-02-24T18:36:22.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 monclient: found mon.noname-b 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.471+0000 7ff489157640 1 -- 10.20.193.18:0/3789170335 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48414c820 con 0x7ff4840c99a0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.472+0000 7ff489157640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 0x7ff48406e540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff478006dc0 tx=0x7ff47800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.472+0000 7ff4727fc640 10 client.?.objecter ms_handle_connect 0x7ff4840c99a0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.472+0000 7ff4727fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478017070 con 0x7ff4840c99a0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 10 monclient: dump: 2026-02-24T18:36:22.492 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:22.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff4727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 5 monclient: authenticate success, global_id 4711 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 10 monclient: _renew_subs 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4840c1e00 con 0x7ff4840c99a0 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 10 client.4711.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 10 monclient: _renew_subs 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4840c2440 con 0x7ff4840c99a0 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.473+0000 7ff48bbe3640 1 librados: init done 2026-02-24T18:36:22.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47800c910 con 0x7ff4840c99a0 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478021060 con 0x7ff4840c99a0 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff473fff640 4 set_mon_vals no callback set 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff4727fc640 10 monclient: dump: 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.495 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff48bbe3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff48bbe3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff48bbe3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.474+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff484150860 con 0x7ff4840c99a0 2026-02-24T18:36:22.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.515+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff47800ccc0 con 0x7ff4840c99a0 2026-02-24T18:36:22.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.516+0000 7ff4727fc640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 0x7ff45407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:22.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.516+0000 7ff489958640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 0x7ff45407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:22.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.516+0000 7ff489958640 10 monclient: get_auth_request con 0x7ff454078e00 auth_method 0 2026-02-24T18:36:22.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.517+0000 7ff489958640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 0x7ff45407b2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff474002770 tx=0x7ff474002a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:22.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.517+0000 7ff4727fc640 10 client.4711.objecter ms_handle_connect 0x7ff454078e00 2026-02-24T18:36:22.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.577+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7ff47805e420 con 0x7ff4840c99a0 2026-02-24T18:36:22.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.577+0000 7ff4727fc640 10 client.4711.objecter ms_dispatch2 0x7ff484000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:22.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.577+0000 7ff4727fc640 3 client.4711.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:22.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.577+0000 7ff4727fc640 3 client.4711.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:22.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.577+0000 7ff4727fc640 20 client.4711.objecter dump_active .. 0 homeless 2026-02-24T18:36:22.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.583+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff4780623e0 con 0x7ff4840c99a0 2026-02-24T18:36:22.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.583+0000 7ff4727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:22.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.583+0000 7ff4727fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:22.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.683+0000 7ff48bbe3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:22.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.683+0000 7ff48bbe3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:22.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.683+0000 7ff48bbe3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:22.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.683+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4840c3ea0 con 0x7ff4840c99a0 2026-02-24T18:36:22.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.686+0000 7ff4727fc640 1 -- 10.20.193.18:0/3789170335 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff4780666e0 con 0x7ff4840c99a0 2026-02-24T18:36:22.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.686+0000 7ff4727fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:22.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.686+0000 7ff4727fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:22.687 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:22.687 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:22.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 10 librados: watch_flush enter 2026-02-24T18:36:22.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 10 librados: watch_flush exit 2026-02-24T18:36:22.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 20 client.4711.objecter shutdown clearing up homeless session... 2026-02-24T18:36:22.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 10 client.4711.objecter successfully canceled tick 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 msgr2=0x7ff45407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 0x7ff45407b2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff474002770 tx=0x7ff474002a30 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 10 monclient: shutdown 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 msgr2=0x7ff48406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 0x7ff48406e540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff478006dc0 tx=0x7ff47800ed70 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 shutdown_connections 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff454078e00 0x7ff45407b2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff4840c99a0 0x7ff48406e540 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff48415a5e0 0x7ff4840c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 --2- 10.20.193.18:0/3789170335 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff48406ff20 0x7ff4841f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 >> 10.20.193.18:0/3789170335 conn(0x7ff4840cd780 msgr2=0x7ff48406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 shutdown_connections 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 -- 10.20.193.18:0/3789170335 wait complete. 2026-02-24T18:36:22.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:22.687+0000 7ff48bbe3640 1 librados: shutdown 2026-02-24T18:36:25.698 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.765+0000 7fca686b0640 10 monclient: get_monmap_and_config 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.765+0000 7fca686b0640 10 monclient: build_initial_monmap 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.765+0000 7fca686b0640 10 monclient: monmap: 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:25.767 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:25.766794+0000 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:25.766794+0000 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:25.768 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 Processor -- start 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 -- start start 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient: init 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient: _add_conns ranks=[2,5,3,4,0,1] 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca6006ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fca600c99a0 0x7fca60070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-c con 0x7fca600c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fca6015a5e0 0x7fca6014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-a con 0x7fca6015a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca600c7e90 con 0x7fca6006ff20 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fca60159a40 con 0x7fca6015a5e0 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fca600c8170 con 0x7fca600c99a0 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:25.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.766+0000 7fca686b0640 10 monclient(hunting): authenticate will time out at 1529.447926s 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fca6015a5e0 0x7fca6014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fca6015a5e0 0x7fca6014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53378/0 (socket says 10.20.193.18:53378) 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 1 -- 10.20.193.18:0/2849340641 learned_addr learned my addr 10.20.193.18:0/2849340641 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): get_auth_request con 0x7fca6015a5e0 auth_method 0 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.767+0000 7fca66c26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 10 monclient(hunting): handle_auth_done global_id 4728 payload 274 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 1 -- 10.20.193.18:0/2849340641 >> v2:10.20.193.110:3300/0 conn(0x7fca600c99a0 msgr2=0x7fca60070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 1 --2- 10.20.193.18:0/2849340641 >> v2:10.20.193.110:3300/0 conn(0x7fca600c99a0 0x7fca60070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 1 -- 10.20.193.18:0/2849340641 >> v1:10.20.193.110:6789/0 conn(0x7fca6006ff20 legacy=0x7fca60159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 10 monclient: _finish_hunting 0 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 1 monclient: found mon.noname-a 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:25.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.768+0000 7fca66c26640 1 -- 10.20.193.18:0/2849340641 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca600c8450 con 0x7fca6015a5e0 2026-02-24T18:36:25.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.769+0000 7fca66c26640 1 --2- 10.20.193.18:0/2849340641 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014fe10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009e50 tx=0x7fca5c02f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caea51ef8fac9e5c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:25.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 1 -- 10.20.193.18:0/2849340641 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c02fcc0 con 0x7fca6015a5e0 2026-02-24T18:36:25.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:25.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:25.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: dump: 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: _finish_auth 0 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: _check_auth_tickets 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 5 monclient: authenticate success, global_id 4728 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 1 -- 10.20.193.18:0/2849340641 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c0373f0 con 0x7fca6015a5e0 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 1 -- 10.20.193.18:0/2849340641 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c02fe60 con 0x7fca6015a5e0 2026-02-24T18:36:25.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca65423640 10 monclient: dump: 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 4 set_mon_vals no callback set 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 10 monclient: shutdown 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 -- 10.20.193.18:0/2849340641 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 msgr2=0x7fca6014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 --2- 10.20.193.18:0/2849340641 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014fe10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009e50 tx=0x7fca5c02f2a0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 -- 10.20.193.18:0/2849340641 shutdown_connections 2026-02-24T18:36:25.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 --2- 10.20.193.18:0/2849340641 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014fe10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 --2- 10.20.193.18:0/2849340641 >> v2:10.20.193.110:3300/0 conn(0x7fca600c99a0 0x7fca60070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 -- 10.20.193.18:0/2849340641 >> 10.20.193.18:0/2849340641 conn(0x7fca600cd780 msgr2=0x7fca600cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 -- 10.20.193.18:0/2849340641 shutdown_connections 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.771+0000 7fca686b0640 1 -- 10.20.193.18:0/2849340641 wait complete. 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient: build_initial_monmap 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient: monmap: 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 librados: starting msgr at 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 librados: starting objecter 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 Processor -- start 2026-02-24T18:36:25.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 -- start start 2026-02-24T18:36:25.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 librados: setting wanted keys 2026-02-24T18:36:25.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 librados: calling monclient init 2026-02-24T18:36:25.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient: init 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca6006ff20 0x7fca601e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-c con 0x7fca6006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-b con 0x7fca600c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): picked mon.noname-a con 0x7fca6015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fca60150350 con 0x7fca6015a5e0 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca600c8a10 con 0x7fca600c99a0 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca601504d0 con 0x7fca6006ff20 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.772+0000 7fca686b0640 10 monclient(hunting): authenticate will time out at 1529.453676s 2026-02-24T18:36:25.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca66c26640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:37996/0 (socket says 10.20.193.18:37996) 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 1 -- 10.20.193.18:0/3237831948 learned_addr learned my addr 10.20.193.18:0/3237831948 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): get_auth_request con 0x7fca600c99a0 auth_method 0 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca65c24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.773+0000 7fca66425640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca6006ff20 0x7fca601e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): get_auth_request con 0x7fca6006ff20 auth_method 0 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.774+0000 7fca66425640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:25.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 10 monclient(hunting): handle_auth_done global_id 4714 payload 824 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 -- 10.20.193.18:0/3237831948 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca6006ff20 msgr2=0x7fca601e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca6006ff20 0x7fca601e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 -- 10.20.193.18:0/3237831948 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 msgr2=0x7fca6014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 10 monclient: _finish_hunting 0 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 monclient: found mon.noname-b 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.775+0000 7fca65c24640 1 -- 10.20.193.18:0/3237831948 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca601f4be0 con 0x7fca600c99a0 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.776+0000 7fca65c24640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b7b0 tx=0x7fca5400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.776+0000 7fca46ffd640 10 client.?.objecter ms_handle_connect 0x7fca600c99a0 2026-02-24T18:36:25.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.776+0000 7fca46ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:25.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54004480 con 0x7fca600c99a0 2026-02-24T18:36:25.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:25.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:25.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: dump: 2026-02-24T18:36:25.782 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: _finish_auth 0 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca54010070 con 0x7fca600c99a0 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 5 monclient: authenticate success, global_id 4714 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _renew_subs 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca601e51c0 con 0x7fca600c99a0 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54002920 con 0x7fca600c99a0 2026-02-24T18:36:25.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca46ffd640 10 monclient: dump: 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca64c22640 4 set_mon_vals no callback set 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 client.4714.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _renew_subs 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601e5780 con 0x7fca600c99a0 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 1 librados: init done 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:25.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:25.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:25.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.778+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca601508a0 con 0x7fca600c99a0 2026-02-24T18:36:25.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.860+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fca5401d030 con 0x7fca600c99a0 2026-02-24T18:36:25.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.861+0000 7fca46ffd640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 0x7fca3007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:25.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.861+0000 7fca66425640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 0x7fca3007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:25.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.861+0000 7fca66425640 10 monclient: get_auth_request con 0x7fca30078e30 auth_method 0 2026-02-24T18:36:25.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.861+0000 7fca46ffd640 10 client.4714.objecter ms_handle_connect 0x7fca30078e30 2026-02-24T18:36:25.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.861+0000 7fca66425640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 0x7fca3007b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca500046e0 tx=0x7fca500091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:25.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.882+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fca5405d470 con 0x7fca600c99a0 2026-02-24T18:36:25.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.882+0000 7fca46ffd640 10 client.4714.objecter ms_dispatch2 0x7fca60000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:25.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.882+0000 7fca46ffd640 3 client.4714.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:25.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.882+0000 7fca46ffd640 3 client.4714.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:25.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.882+0000 7fca46ffd640 20 client.4714.objecter dump_active .. 0 homeless 2026-02-24T18:36:25.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.887+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fca54023020 con 0x7fca600c99a0 2026-02-24T18:36:25.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.887+0000 7fca46ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:25.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.887+0000 7fca46ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:25.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.987+0000 7fca686b0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:25.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.987+0000 7fca686b0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:25.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.987+0000 7fca686b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:36:25.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.987+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fca601e6a80 con 0x7fca600c99a0 2026-02-24T18:36:25.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.990+0000 7fca46ffd640 1 -- 10.20.193.18:0/3237831948 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fca601f4be0 con 0x7fca600c99a0 2026-02-24T18:36:25.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.990+0000 7fca46ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:25.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.990+0000 7fca46ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:25.992 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:25.992 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:25.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 10 librados: watch_flush enter 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 10 librados: watch_flush exit 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 20 client.4714.objecter shutdown clearing up homeless session... 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 10 client.4714.objecter successfully canceled tick 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 msgr2=0x7fca3007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 0x7fca3007b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca500046e0 tx=0x7fca500091c0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 10 monclient: shutdown 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 msgr2=0x7fca60143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b7b0 tx=0x7fca5400bc80 comp rx=0 tx=0).stop 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 shutdown_connections 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fca30078e30 0x7fca3007b2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca600c99a0 0x7fca60143740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca6015a5e0 0x7fca6014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 --2- 10.20.193.18:0/3237831948 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca6006ff20 0x7fca601e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 >> 10.20.193.18:0/3237831948 conn(0x7fca600cd780 msgr2=0x7fca600c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 shutdown_connections 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 -- 10.20.193.18:0/3237831948 wait complete. 2026-02-24T18:36:25.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:25.991+0000 7fca686b0640 1 librados: shutdown 2026-02-24T18:36:29.002 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 10 monclient: get_monmap_and_config 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 10 monclient: build_initial_monmap 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 10 monclient: monmap: 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:29.072063+0000 2026-02-24T18:36:29.072 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:29.072063+0000 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 1 Processor -- start 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 1 -- start start 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.071+0000 7f8b025fe640 10 monclient: init 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient: _add_conns ranks=[5,3,2,0,4,1] 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f8afc1527d0 0x7f8afc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-c con 0x7f8afc1527d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8afc152bb0 0x7f8afc0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-a con 0x7f8afc152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8afc0c5fc0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc06e300 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:29.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc06f1f0 con 0x7f8afc152bb0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc0c6470 con 0x7f8afc1527d0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8b025fe640 10 monclient(hunting): authenticate will time out at 1532.753211s 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8afc152bb0 0x7f8afc0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8afc152bb0 0x7f8afc0c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55690/0 (socket says 10.20.193.18:55690) 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 1 -- 10.20.193.18:0/2588940717 learned_addr learned my addr 10.20.193.18:0/2588940717 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): get_auth_request con 0x7f8afc152bb0 auth_method 0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afbfff640 1 --2- 10.20.193.18:0/2588940717 >> v2:10.20.193.110:3300/0 conn(0x7f8afc1527d0 0x7f8afc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.072+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): get_auth_request con 0x7f8afc1527d0 auth_method 0 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:29.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_done global_id 4731 payload 274 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 1 -- 10.20.193.18:0/2588940717 >> v2:10.20.193.110:3300/0 conn(0x7f8afc1527d0 msgr2=0x7f8afc06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 1 --2- 10.20.193.18:0/2588940717 >> v2:10.20.193.110:3300/0 conn(0x7f8afc1527d0 0x7f8afc06da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 1 -- 10.20.193.18:0/2588940717 >> v1:10.20.193.110:6789/0 conn(0x7f8afc0c5fc0 legacy=0x7f8afc0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 1 monclient: found mon.noname-a 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:29.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.073+0000 7f8afb7fe640 1 -- 10.20.193.18:0/2588940717 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc06e5e0 con 0x7f8afc152bb0 2026-02-24T18:36:29.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.074+0000 7f8afb7fe640 1 --2- 10.20.193.18:0/2588940717 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc152bb0 0x7f8afc0c5a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0caec0 tx=0x7f8aec02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6002faaf4725ae50 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 1 -- 10.20.193.18:0/2588940717 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec03c070 con 0x7f8afc152bb0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: dump: 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: _finish_auth 0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 1 -- 10.20.193.18:0/2588940717 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aec02fbe0 con 0x7f8afc152bb0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 1 -- 10.20.193.18:0/2588940717 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec037440 con 0x7f8afc152bb0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 5 monclient: authenticate success, global_id 4731 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 4 set_mon_vals no callback set 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:29.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8afaffd640 10 monclient: dump: 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 10 monclient: shutdown 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 1 -- 10.20.193.18:0/2588940717 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc152bb0 msgr2=0x7f8afc0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.076+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2588940717 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc152bb0 0x7f8afc0c5a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0caec0 tx=0x7f8aec02eed0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 -- 10.20.193.18:0/2588940717 shutdown_connections 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2588940717 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc152bb0 0x7f8afc0c5a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2588940717 >> v2:10.20.193.110:3300/0 conn(0x7f8afc1527d0 0x7f8afc06da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 -- 10.20.193.18:0/2588940717 >> 10.20.193.18:0/2588940717 conn(0x7f8afc0cd780 msgr2=0x7f8afc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 -- 10.20.193.18:0/2588940717 shutdown_connections 2026-02-24T18:36:29.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 -- 10.20.193.18:0/2588940717 wait complete. 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 10 monclient: build_initial_monmap 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 10 monclient: monmap: 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 librados: starting msgr at 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 librados: starting objecter 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 Processor -- start 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 -- start start 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 librados: setting wanted keys 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 1 librados: calling monclient init 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.077+0000 7f8b025fe640 10 monclient: init 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-c con 0x7f8afc0c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc1527d0 0x7f8afc154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-a con 0x7f8afc1527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8afc152bb0 0x7f8afc155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): picked mon.noname-b con 0x7f8afc152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc06ee80 con 0x7f8afc1527d0 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc06eba0 con 0x7f8afc152bb0 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc06dfa0 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8b025fe640 10 monclient(hunting): authenticate will time out at 1532.759135s 2026-02-24T18:36:29.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:43138/0 (socket says 10.20.193.18:43138) 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 1 -- 10.20.193.18:0/2194017245 learned_addr learned my addr 10.20.193.18:0/2194017245 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 10 monclient(hunting): get_auth_request con 0x7f8afc0c5fc0 auth_method 0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.078+0000 7f8afb7fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc1527d0 0x7f8afc154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8b00b74640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8afc152bb0 0x7f8afc155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): get_auth_request con 0x7f8afc1527d0 auth_method 0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8b00b74640 10 monclient(hunting): get_auth_request con 0x7f8afc152bb0 auth_method 0 2026-02-24T18:36:29.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8b00b74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8b00b74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.079+0000 7f8b00b74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8b00b74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8b00b74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8b00b74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 10 monclient(hunting): handle_auth_done global_id 4688 payload 824 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 -- 10.20.193.18:0/2194017245 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8afc152bb0 msgr2=0x7f8afc155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8afc152bb0 0x7f8afc155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 -- 10.20.193.18:0/2194017245 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc1527d0 msgr2=0x7f8afc154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc1527d0 0x7f8afc154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 monclient: found mon.noname-c 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:36:29.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.080+0000 7f8afbfff640 1 -- 10.20.193.18:0/2194017245 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc165bd0 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.082+0000 7f8afbfff640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800ec10 tx=0x7f8ae800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:29.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.082+0000 7f8af8ff9640 10 client.?.objecter ms_handle_connect 0x7f8afc0c5fc0 2026-02-24T18:36:29.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.082+0000 7f8af8ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8017070 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: dump: 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: _finish_auth 0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 5 monclient: authenticate success, global_id 4688 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae800cc40 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8013470 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:29.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8afa7fc640 4 set_mon_vals no callback set 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8af8ff9640 10 monclient: dump: 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 10 monclient: _renew_subs 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc165610 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 10 client.4688.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 10 monclient: _renew_subs 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc154150 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.084+0000 7f8b025fe640 1 librados: init done 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.085+0000 7f8b025fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.085+0000 7f8b025fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.085+0000 7f8b025fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:29.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.085+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8afc06da60 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.096+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8ae8013630 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.097+0000 7f8af8ff9640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 0x7f8acc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:29.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.097+0000 7f8afb7fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 0x7f8acc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:29.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.097+0000 7f8afb7fe640 10 monclient: get_auth_request con 0x7f8acc078e00 auth_method 0 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 10 client.4688.objecter ms_handle_connect 0x7f8acc078e00 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8afb7fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 0x7f8acc07b2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8afc158940 tx=0x7f8aec039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f8ae805e520 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 10 client.4688.objecter ms_dispatch2 0x7f8afc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 3 client.4688.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 3 client.4688.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:29.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.098+0000 7f8af8ff9640 20 client.4688.objecter dump_active .. 0 homeless 2026-02-24T18:36:29.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.196+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8ae80624e0 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.196+0000 7f8af8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:29.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.196+0000 7f8af8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:29.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.297+0000 7f8b025fe640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:29.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.297+0000 7f8b025fe640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:29.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.297+0000 7f8b025fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:29.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.297+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8afc074d20 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.300+0000 7f8af8ff9640 1 -- 10.20.193.18:0/2194017245 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8ae80667e0 con 0x7f8afc0c5fc0 2026-02-24T18:36:29.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.300+0000 7f8af8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:29.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.300+0000 7f8af8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:29.302 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:29.302 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 10 librados: watch_flush enter 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 10 librados: watch_flush exit 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 20 client.4688.objecter shutdown clearing up homeless session... 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 10 client.4688.objecter successfully canceled tick 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 msgr2=0x7f8acc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 0x7f8acc07b2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8afc158940 tx=0x7f8aec039040 comp rx=0 tx=0).stop 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 10 monclient: shutdown 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 msgr2=0x7f8afc1f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:29.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800ec10 tx=0x7f8ae800c520 comp rx=0 tx=0).stop 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 shutdown_connections 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8acc078e00 0x7f8acc07b2c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8afc152bb0 0x7f8afc155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8afc1527d0 0x7f8afc154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 --2- 10.20.193.18:0/2194017245 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8afc0c5fc0 0x7f8afc1f7530 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 >> 10.20.193.18:0/2194017245 conn(0x7f8afc0cd780 msgr2=0x7f8afc0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 shutdown_connections 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 -- 10.20.193.18:0/2194017245 wait complete. 2026-02-24T18:36:29.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:29.301+0000 7f8b025fe640 1 librados: shutdown 2026-02-24T18:36:32.312 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:32.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.382+0000 7f91f566d640 10 monclient: get_monmap_and_config 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.382+0000 7f91f566d640 10 monclient: build_initial_monmap 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.382+0000 7f91f566d640 10 monclient: monmap: 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:32.383416+0000 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:32.383416+0000 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:32.385 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.382+0000 7f91f566d640 1 Processor -- start 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 -- start start 2026-02-24T18:36:32.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient: init 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient: _add_conns ranks=[5,3,2,4,1,0] 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f91f006ff20 0x7f91f0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-c con 0x7f91f006ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f91f00c99a0 0x7f91f0159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-a con 0x7f91f00c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f91f015a410 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f015a860 con 0x7f91f015a410 2026-02-24T18:36:32.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f91f0159a40 con 0x7f91f00c99a0 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f91f00c7e90 con 0x7f91f006ff20 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91f566d640 10 monclient(hunting): authenticate will time out at 1536.064545s 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91ee7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f91f00c99a0 0x7f91f0159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.383+0000 7f91eeffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f91f006ff20 0x7f91f0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f91f00c99a0 0x7f91f0159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55706/0 (socket says 10.20.193.18:55706) 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 1 -- 10.20.193.18:0/618803607 learned_addr learned my addr 10.20.193.18:0/618803607 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): get_auth_request con 0x7f91f00c99a0 auth_method 0 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:32.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f006ff20 auth_method 0 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.384+0000 7f91eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_done global_id 4737 payload 274 2026-02-24T18:36:32.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 1 -- 10.20.193.18:0/618803607 >> v2:10.20.193.110:3300/0 conn(0x7f91f006ff20 msgr2=0x7f91f0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 1 --2- 10.20.193.18:0/618803607 >> v2:10.20.193.110:3300/0 conn(0x7f91f006ff20 0x7f91f0070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 1 -- 10.20.193.18:0/618803607 >> v1:10.20.193.110:6789/0 conn(0x7f91f015a410 legacy=0x7f91f014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 1 monclient: found mon.noname-a 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.385+0000 7f91ee7fc640 1 -- 10.20.193.18:0/618803607 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f00c8170 con 0x7f91f00c99a0 2026-02-24T18:36:32.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.386+0000 7f91ee7fc640 1 --2- 10.20.193.18:0/618803607 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f00c99a0 0x7f91f0159ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009970 tx=0x7f91d802eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54586ffba2df6e50 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 1 -- 10.20.193.18:0/618803607 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d803c070 con 0x7f91f00c99a0 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: dump: 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:32.399 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: _finish_auth 0 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: _check_auth_tickets 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 1 -- 10.20.193.18:0/618803607 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d802fbc0 con 0x7f91f00c99a0 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 5 monclient: authenticate success, global_id 4737 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:32.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 1 -- 10.20.193.18:0/618803607 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8037440 con 0x7f91f00c99a0 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 4 set_mon_vals no callback set 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91edffb640 10 monclient: dump: 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:32.401 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 10 monclient: shutdown 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 -- 10.20.193.18:0/618803607 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f00c99a0 msgr2=0x7f91f0159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 --2- 10.20.193.18:0/618803607 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f00c99a0 0x7f91f0159ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009970 tx=0x7f91d802eef0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 -- 10.20.193.18:0/618803607 shutdown_connections 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 --2- 10.20.193.18:0/618803607 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f00c99a0 0x7f91f0159ed0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 --2- 10.20.193.18:0/618803607 >> v2:10.20.193.110:3300/0 conn(0x7f91f006ff20 0x7f91f0070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 -- 10.20.193.18:0/618803607 >> 10.20.193.18:0/618803607 conn(0x7f91f00cd780 msgr2=0x7f91f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:32.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 -- 10.20.193.18:0/618803607 shutdown_connections 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.388+0000 7f91f566d640 1 -- 10.20.193.18:0/618803607 wait complete. 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient: build_initial_monmap 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient: monmap: 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:32.403 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 librados: starting msgr at 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 librados: starting objecter 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 Processor -- start 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 -- start start 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 librados: setting wanted keys 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 librados: calling monclient init 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient: init 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f91f006ff20 0x7f91f01f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-b con 0x7f91f006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f91f00c99a0 0x7f91f01f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-c con 0x7f91f00c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): picked mon.noname-a con 0x7f91f015a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91f00c8a10 con 0x7f91f015a410 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91f00c8730 con 0x7f91f006ff20 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0150520 con 0x7f91f00c99a0 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.389+0000 7f91f566d640 10 monclient(hunting): authenticate will time out at 1536.070921s 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f91f00c99a0 0x7f91f01f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55708/0 (socket says 10.20.193.18:55708) 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 1 -- 10.20.193.18:0/138850095 learned_addr learned my addr 10.20.193.18:0/138850095 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request con 0x7f91f015a410 auth_method 0 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): get_auth_request con 0x7f91f00c99a0 auth_method 0 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:32.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91eeffd640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f91f006ff20 0x7f91f01f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91eeffd640 10 monclient(hunting): get_auth_request con 0x7f91f006ff20 auth_method 0 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91eeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.390+0000 7f91eeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 10 monclient(hunting): handle_auth_done global_id 4740 payload 824 2026-02-24T18:36:32.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 -- 10.20.193.18:0/138850095 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f91f00c99a0 msgr2=0x7f91f01f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f91f00c99a0 0x7f91f01f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 -- 10.20.193.18:0/138850095 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f91f006ff20 msgr2=0x7f91f01f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f91f006ff20 0x7f91f01f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 monclient: found mon.noname-a 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.391+0000 7f91ef7fe640 1 -- 10.20.193.18:0/138850095 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f01f51d0 con 0x7f91f015a410 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.392+0000 7f91ef7fe640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91e000cbd0 tx=0x7f91e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.392+0000 7f91cf7fe640 10 client.?.objecter ms_handle_connect 0x7f91f015a410 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.392+0000 7f91cf7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e000ad50 con 0x7f91f015a410 2026-02-24T18:36:32.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: dump: 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: _finish_auth 0 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e0003a70 con 0x7f91f015a410 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 5 monclient: authenticate success, global_id 4740 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 10 monclient: _renew_subs 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01f54b0 con 0x7f91f015a410 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003e40 con 0x7f91f015a410 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:32.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91cf7fe640 10 monclient: dump: 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91ed7fa640 4 set_mon_vals no callback set 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 10 client.4740.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:32.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 10 monclient: _renew_subs 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01f5a70 con 0x7f91f015a410 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.394+0000 7f91f566d640 1 librados: init done 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.395+0000 7f91f566d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.395+0000 7f91f566d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.395+0000 7f91f566d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.395+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f0070300 con 0x7f91f015a410 2026-02-24T18:36:32.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.502+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f91e00050d0 con 0x7f91f015a410 2026-02-24T18:36:32.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.503+0000 7f91cf7fe640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 0x7f91bc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:32.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.503+0000 7f91eeffd640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 0x7f91bc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:32.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.503+0000 7f91eeffd640 10 monclient: get_auth_request con 0x7f91bc078e50 auth_method 0 2026-02-24T18:36:32.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.503+0000 7f91cf7fe640 10 client.4740.objecter ms_handle_connect 0x7f91bc078e50 2026-02-24T18:36:32.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.503+0000 7f91eeffd640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 0x7f91bc07b310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c40 tx=0x7f91e4009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f91e005e670 con 0x7f91f015a410 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 10 client.4740.objecter ms_dispatch2 0x7f91f0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 3 client.4740.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 3 client.4740.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 20 client.4740.objecter dump_active .. 0 homeless 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f91e0062630 con 0x7f91f015a410 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:32.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.511+0000 7f91cf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:32.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.614+0000 7f91f566d640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:32.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.614+0000 7f91f566d640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:32.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.614+0000 7f91f566d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:32.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.614+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f91f014feb0 con 0x7f91f015a410 2026-02-24T18:36:32.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.617+0000 7f91cf7fe640 1 -- 10.20.193.18:0/138850095 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f91e0066930 con 0x7f91f015a410 2026-02-24T18:36:32.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.617+0000 7f91cf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:32.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.617+0000 7f91cf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:32.620 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:32.620 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:32.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 10 librados: watch_flush enter 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 10 librados: watch_flush exit 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 20 client.4740.objecter shutdown clearing up homeless session... 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 10 client.4740.objecter successfully canceled tick 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 msgr2=0x7f91bc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 0x7f91bc07b310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c40 tx=0x7f91e4009290 comp rx=0 tx=0).stop 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 10 monclient: shutdown 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 msgr2=0x7f91f01f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.618+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91e000cbd0 tx=0x7f91e000a430 comp rx=0 tx=0).stop 2026-02-24T18:36:32.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 shutdown_connections 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f91bc078e50 0x7f91bc07b310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f91f015a410 0x7f91f01f6550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f91f00c99a0 0x7f91f01f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 --2- 10.20.193.18:0/138850095 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f91f006ff20 0x7f91f01f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 >> 10.20.193.18:0/138850095 conn(0x7f91f00cd780 msgr2=0x7f91f00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 shutdown_connections 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 -- 10.20.193.18:0/138850095 wait complete. 2026-02-24T18:36:32.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:32.619+0000 7f91f566d640 1 librados: shutdown 2026-02-24T18:36:35.631 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:35.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.698+0000 7fb6644f5640 10 monclient: get_monmap_and_config 2026-02-24T18:36:35.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.698+0000 7fb6644f5640 10 monclient: build_initial_monmap 2026-02-24T18:36:35.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.698+0000 7fb6644f5640 10 monclient: monmap: 2026-02-24T18:36:35.701 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:35.700613+0000 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:35.700613+0000 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:35.702 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 Processor -- start 2026-02-24T18:36:35.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 -- start start 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient: init 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient: _add_conns ranks=[3,2,4,1,5,0] 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb65c1527d0 0x7fb65c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-a con 0x7fb65c1527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb65c152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fb65c0c6190 0x7fb65c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-b con 0x7fb65c0c6190 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65c06e300 con 0x7fb65c152bb0 2026-02-24T18:36:35.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb65c06f1f0 con 0x7fb65c1527d0 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb65c06e5e0 con 0x7fb65c0c6190 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.699+0000 7fb6644f5640 10 monclient(hunting): authenticate will time out at 1539.381853s 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb65c1527d0 0x7fb65c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fb65c0c6190 0x7fb65c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb661a69640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fb65c152bb0 0x7fb65c0c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:55152/0 (socket says 10.20.193.18:55152) 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb661a69640 1 -- 10.20.193.18:0/734586937 learned_addr learned my addr 10.20.193.18:0/734586937 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): get_auth_request con 0x7fb65c1527d0 auth_method 0 2026-02-24T18:36:35.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): get_auth_request con 0x7fb65c0c6190 auth_method 0 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.700+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:35.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 10 monclient(hunting): handle_auth_done global_id 4743 payload 274 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 1 -- 10.20.193.18:0/734586937 >> v2:10.20.193.32:3300/0 conn(0x7fb65c0c6190 msgr2=0x7fb65c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 1 --2- 10.20.193.18:0/734586937 >> v2:10.20.193.32:3300/0 conn(0x7fb65c0c6190 0x7fb65c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 1 -- 10.20.193.18:0/734586937 >> v1:10.20.193.110:6789/0 conn(0x7fb65c152bb0 legacy=0x7fb65c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 10 monclient: _finish_hunting 0 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 1 monclient: found mon.noname-a 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.701+0000 7fb66226a640 1 -- 10.20.193.18:0/734586937 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c06e8c0 con 0x7fb65c1527d0 2026-02-24T18:36:35.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.702+0000 7fb66226a640 1 --2- 10.20.193.18:0/734586937 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c1527d0 0x7fb65c06da60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb65c158940 tx=0x7fb64c02ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e111cf842c5a4128 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:35.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 1 -- 10.20.193.18:0/734586937 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c03c070 con 0x7fb65c1527d0 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: dump: 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:35.716 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: _finish_auth 0 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: _check_auth_tickets 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 1 -- 10.20.193.18:0/734586937 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c02faf0 con 0x7fb65c1527d0 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 1 -- 10.20.193.18:0/734586937 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c037440 con 0x7fb65c1527d0 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 5 monclient: authenticate success, global_id 4743 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 4 set_mon_vals no callback set 2026-02-24T18:36:35.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb661268640 10 monclient: dump: 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:35.718 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 10 monclient: shutdown 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 -- 10.20.193.18:0/734586937 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c1527d0 msgr2=0x7fb65c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 --2- 10.20.193.18:0/734586937 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c1527d0 0x7fb65c06da60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb65c158940 tx=0x7fb64c02ede0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 -- 10.20.193.18:0/734586937 shutdown_connections 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 --2- 10.20.193.18:0/734586937 >> v2:10.20.193.32:3300/0 conn(0x7fb65c0c6190 0x7fb65c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 --2- 10.20.193.18:0/734586937 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c1527d0 0x7fb65c06da60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.704+0000 7fb6644f5640 1 -- 10.20.193.18:0/734586937 >> 10.20.193.18:0/734586937 conn(0x7fb65c0cd780 msgr2=0x7fb65c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 -- 10.20.193.18:0/734586937 shutdown_connections 2026-02-24T18:36:35.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 -- 10.20.193.18:0/734586937 wait complete. 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 10 monclient: build_initial_monmap 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 10 monclient: monmap: 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:35.720 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 librados: starting msgr at 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 librados: starting objecter 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 Processor -- start 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 -- start start 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 librados: setting wanted keys 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 1 librados: calling monclient init 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.705+0000 7fb6644f5640 10 monclient: init 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-a con 0x7fb65c0c6190 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb65c1527d0 0x7fb65c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-c con 0x7fb65c1527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb65c152bb0 0x7fb65c073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): picked mon.noname-b con 0x7fb65c152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c06dfa0 con 0x7fb65c0c6190 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c06ee80 con 0x7fb65c152bb0 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c0c93a0 con 0x7fb65c1527d0 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb6644f5640 10 monclient(hunting): authenticate will time out at 1539.388507s 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55730/0 (socket says 10.20.193.18:55730) 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 1 -- 10.20.193.18:0/2030084185 learned_addr learned my addr 10.20.193.18:0/2030084185 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 10 monclient(hunting): get_auth_request con 0x7fb65c0c6190 auth_method 0 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb66226a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb662a6b640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb65c152bb0 0x7fb65c073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.706+0000 7fb661a69640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb65c1527d0 0x7fb65c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb66226a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): get_auth_request con 0x7fb65c152bb0 auth_method 0 2026-02-24T18:36:35.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): get_auth_request con 0x7fb65c1527d0 auth_method 0 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb662a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.707+0000 7fb661a69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:35.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 10 monclient(hunting): handle_auth_done global_id 4746 payload 824 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 -- 10.20.193.18:0/2030084185 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb65c1527d0 msgr2=0x7fb65c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb65c1527d0 0x7fb65c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 -- 10.20.193.18:0/2030084185 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb65c152bb0 msgr2=0x7fb65c073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb65c152bb0 0x7fb65c073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 10 monclient: _finish_hunting 0 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 monclient: found mon.noname-a 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.708+0000 7fb66226a640 1 -- 10.20.193.18:0/2030084185 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c0c4200 con 0x7fb65c0c6190 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.709+0000 7fb66226a640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb64c03a040 tx=0x7fb64c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.709+0000 7fb642ffd640 10 client.?.objecter ms_handle_connect 0x7fb65c0c6190 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.709+0000 7fb642ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:35.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c03c040 con 0x7fb65c0c6190 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: dump: 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:35.726 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: _finish_auth 0 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c042610 con 0x7fb65c0c6190 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0429e0 con 0x7fb65c0c6190 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb642ffd640 10 monclient: dump: 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:35.727 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 5 monclient: authenticate success, global_id 4746 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb660a67640 4 set_mon_vals no callback set 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _renew_subs 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c0c3690 con 0x7fb65c0c6190 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 client.4746.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _renew_subs 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c0c3c40 con 0x7fb65c0c6190 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 1 librados: init done 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.711+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c0c9780 con 0x7fb65c0c6190 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.723+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb64c04b450 con 0x7fb65c0c6190 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.723+0000 7fb642ffd640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 0x7fb62c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.724+0000 7fb661a69640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 0x7fb62c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.724+0000 7fb661a69640 10 monclient: get_auth_request con 0x7fb62c078e50 auth_method 0 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.724+0000 7fb661a69640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 0x7fb62c07b310 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb65c1495c0 tx=0x7fb650009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.724+0000 7fb642ffd640 10 client.4746.objecter ms_handle_connect 0x7fb62c078e50 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.725+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fb64c02fa70 con 0x7fb65c0c6190 2026-02-24T18:36:35.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.725+0000 7fb642ffd640 10 client.4746.objecter ms_dispatch2 0x7fb65c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:35.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.725+0000 7fb642ffd640 3 client.4746.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:35.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.725+0000 7fb642ffd640 3 client.4746.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:35.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.725+0000 7fb642ffd640 20 client.4746.objecter dump_active .. 0 homeless 2026-02-24T18:36:35.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.825+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb64c086880 con 0x7fb65c0c6190 2026-02-24T18:36:35.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.825+0000 7fb642ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:35.828 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.825+0000 7fb642ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.927+0000 7fb6644f5640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.927+0000 7fb6644f5640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.927+0000 7fb6644f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:35.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.927+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb65c0cb680 con 0x7fb65c0c6190 2026-02-24T18:36:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.929+0000 7fb642ffd640 1 -- 10.20.193.18:0/2030084185 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb64c08ab80 con 0x7fb65c0c6190 2026-02-24T18:36:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.929+0000 7fb642ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:35.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.929+0000 7fb642ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:35.933 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:35.933 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 10 librados: watch_flush enter 2026-02-24T18:36:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 10 librados: watch_flush exit 2026-02-24T18:36:35.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 20 client.4746.objecter shutdown clearing up homeless session... 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 10 client.4746.objecter successfully canceled tick 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 msgr2=0x7fb62c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 0x7fb62c07b310 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb65c1495c0 tx=0x7fb650009040 comp rx=0 tx=0).stop 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 10 monclient: shutdown 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 msgr2=0x7fb65c0c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb64c03a040 tx=0x7fb64c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 shutdown_connections 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb62c078e50 0x7fb62c07b310 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb65c152bb0 0x7fb65c073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb65c1527d0 0x7fb65c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 --2- 10.20.193.18:0/2030084185 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb65c0c6190 0x7fb65c0c3150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 >> 10.20.193.18:0/2030084185 conn(0x7fb65c0cd780 msgr2=0x7fb65c0749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 shutdown_connections 2026-02-24T18:36:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 -- 10.20.193.18:0/2030084185 wait complete. 2026-02-24T18:36:35.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:35.930+0000 7fb6644f5640 1 librados: shutdown 2026-02-24T18:36:38.943 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.011+0000 7f68b0386640 10 monclient: get_monmap_and_config 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.011+0000 7f68b0386640 10 monclient: build_initial_monmap 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.011+0000 7f68b0386640 10 monclient: monmap: 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:39.012778+0000 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:39.012778+0000 2026-02-24T18:36:39.014 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 Processor -- start 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 -- start start 2026-02-24T18:36:39.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient: init 2026-02-24T18:36:39.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:39.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient: _add_conns ranks=[4,3,5,1,2,0] 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f68a806ff20 0x7f68a8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-b con 0x7f68a806ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f68a80c99a0 0x7f68a8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-a con 0x7f68a80c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f68a815a410 0x7f68a814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-c con 0x7f68a815a410 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f68a8159a40 con 0x7f68a80c99a0 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f68a815a860 con 0x7f68a806ff20 2026-02-24T18:36:39.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f68a80c7e90 con 0x7f68a815a410 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.012+0000 7f68b0386640 10 monclient(hunting): authenticate will time out at 1542.693915s 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f68a80c99a0 0x7f68a8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f68a80c99a0 0x7f68a8159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55746/0 (socket says 10.20.193.18:55746) 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 1 -- 10.20.193.18:0/3625547971 learned_addr learned my addr 10.20.193.18:0/3625547971 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): get_auth_request con 0x7f68a80c99a0 auth_method 0 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ae0fb640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.32:3300/0 conn(0x7f68a806ff20 0x7f68a8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ae0fb640 10 monclient(hunting): get_auth_request con 0x7f68a806ff20 auth_method 0 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ae0fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ae0fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ae0fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.013+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae8fc640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.110:3300/0 conn(0x7f68a815a410 0x7f68a814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae8fc640 10 monclient(hunting): get_auth_request con 0x7f68a815a410 auth_method 0 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae8fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae8fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ae8fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_done global_id 4749 payload 274 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 -- 10.20.193.18:0/3625547971 >> v2:10.20.193.110:3300/0 conn(0x7f68a815a410 msgr2=0x7f68a814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.110:3300/0 conn(0x7f68a815a410 0x7f68a814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 -- 10.20.193.18:0/3625547971 >> v2:10.20.193.32:3300/0 conn(0x7f68a806ff20 msgr2=0x7f68a8070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.32:3300/0 conn(0x7f68a806ff20 0x7f68a8070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 10 monclient: _finish_hunting 0 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 monclient: found mon.noname-a 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.014+0000 7f68ad8fa640 1 -- 10.20.193.18:0/3625547971 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a80c8170 con 0x7f68a80c99a0 2026-02-24T18:36:39.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.015+0000 7f68ad8fa640 1 --2- 10.20.193.18:0/3625547971 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a80c99a0 0x7f68a8159ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f68a80c5380 tx=0x7f689c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b28ef246c368437 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 1 -- 10.20.193.18:0/3625547971 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c03c070 con 0x7f68a80c99a0 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: dump: 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.029 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: _finish_auth 0 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: _check_auth_tickets 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 1 -- 10.20.193.18:0/3625547971 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689c02fb50 con 0x7f68a80c99a0 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:39.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 1 -- 10.20.193.18:0/3625547971 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c037440 con 0x7f68a80c99a0 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 5 monclient: authenticate success, global_id 4749 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68ad0f9640 10 monclient: dump: 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:39.031 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 4 set_mon_vals no callback set 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 10 monclient: shutdown 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 1 -- 10.20.193.18:0/3625547971 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a80c99a0 msgr2=0x7f68a8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.017+0000 7f68b0386640 1 --2- 10.20.193.18:0/3625547971 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a80c99a0 0x7f68a8159ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f68a80c5380 tx=0x7f689c02eed0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 -- 10.20.193.18:0/3625547971 shutdown_connections 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.110:3300/0 conn(0x7f68a815a410 0x7f68a814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 --2- 10.20.193.18:0/3625547971 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a80c99a0 0x7f68a8159ed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 --2- 10.20.193.18:0/3625547971 >> v2:10.20.193.32:3300/0 conn(0x7f68a806ff20 0x7f68a8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 -- 10.20.193.18:0/3625547971 >> 10.20.193.18:0/3625547971 conn(0x7f68a80cd780 msgr2=0x7f68a80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 -- 10.20.193.18:0/3625547971 shutdown_connections 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 -- 10.20.193.18:0/3625547971 wait complete. 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 10 monclient: build_initial_monmap 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 10 monclient: monmap: 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:39.033 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 librados: starting msgr at 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 librados: starting objecter 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.018+0000 7f68b0386640 1 Processor -- start 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 -- start start 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 librados: setting wanted keys 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 librados: calling monclient init 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient: init 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f68a806ff20 0x7f68a8149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-c con 0x7f68a806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f68a80c99a0 0x7f68a81eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-b con 0x7f68a80c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): picked mon.noname-a con 0x7f68a815a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80c8a10 con 0x7f68a815a410 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80c8730 con 0x7f68a80c99a0 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8150350 con 0x7f68a806ff20 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68b0386640 10 monclient(hunting): authenticate will time out at 1542.700413s 2026-02-24T18:36:39.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55750/0 (socket says 10.20.193.18:55750) 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 1 -- 10.20.193.18:0/3112445635 learned_addr learned my addr 10.20.193.18:0/3112445635 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): get_auth_request con 0x7f68a815a410 auth_method 0 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae0fb640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f68a806ff20 0x7f68a8149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae0fb640 10 monclient(hunting): get_auth_request con 0x7f68a806ff20 auth_method 0 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ad8fa640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f68a80c99a0 0x7f68a81eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae0fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae0fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae0fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.019+0000 7f68ae8fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): get_auth_request con 0x7f68a80c99a0 auth_method 0 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ae0fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.020+0000 7f68ad8fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:39.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 10 monclient(hunting): handle_auth_done global_id 4752 payload 824 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 -- 10.20.193.18:0/3112445635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f68a806ff20 msgr2=0x7f68a8149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f68a806ff20 0x7f68a8149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 -- 10.20.193.18:0/3112445635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f68a80c99a0 msgr2=0x7f68a81eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f68a80c99a0 0x7f68a81eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 10 monclient: _finish_hunting 0 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 monclient: found mon.noname-a 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.021+0000 7f68ae8fc640 1 -- 10.20.193.18:0/3112445635 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a814a030 con 0x7f68a815a410 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.022+0000 7f68ae8fc640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68a4007e60 tx=0x7f68a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.022+0000 7f688effd640 10 client.?.objecter ms_handle_connect 0x7f68a815a410 2026-02-24T18:36:39.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.022+0000 7f688effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4014420 con 0x7f68a815a410 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: dump: 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:39.039 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: _finish_auth 0 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a4019070 con 0x7f68a815a410 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4014800 con 0x7f68a815a410 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f688effd640 10 monclient: dump: 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:39.040 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f68ac8f8640 4 set_mon_vals no callback set 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f68b0386640 5 monclient: authenticate success, global_id 4752 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f68b0386640 10 monclient: _renew_subs 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f68b0386640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.023+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a814a300 con 0x7f68a815a410 2026-02-24T18:36:39.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 client.4752.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 monclient: _renew_subs 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a814a8c0 con 0x7f68a815a410 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 1 librados: init done 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.024+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a81506e0 con 0x7f68a815a410 2026-02-24T18:36:39.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.129+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f68a40149a0 con 0x7f68a815a410 2026-02-24T18:36:39.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.130+0000 7f688effd640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 0x7f687807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:39.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.130+0000 7f68ae0fb640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 0x7f687807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:39.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.130+0000 7f68ae0fb640 10 monclient: get_auth_request con 0x7f6878078e80 auth_method 0 2026-02-24T18:36:39.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.131+0000 7f68ae0fb640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 0x7f687807b340 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6898009c40 tx=0x7f6898009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:39.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.131+0000 7f688effd640 10 client.4752.objecter ms_handle_connect 0x7f6878078e80 2026-02-24T18:36:39.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f68a405eec0 con 0x7f68a815a410 2026-02-24T18:36:39.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 10 client.4752.objecter ms_dispatch2 0x7f68a8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:39.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 3 client.4752.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:39.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 3 client.4752.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:39.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 20 client.4752.objecter dump_active .. 0 homeless 2026-02-24T18:36:39.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f68a4062e80 con 0x7f68a815a410 2026-02-24T18:36:39.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:39.141 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.139+0000 7f688effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:39.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.239+0000 7f68b0386640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:39.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.239+0000 7f68b0386640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:39.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.239+0000 7f68b0386640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:39.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.239+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f68a814d5a0 con 0x7f68a815a410 2026-02-24T18:36:39.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.241+0000 7f688effd640 1 -- 10.20.193.18:0/3112445635 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f68a4067180 con 0x7f68a815a410 2026-02-24T18:36:39.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.241+0000 7f688effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:39.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.241+0000 7f688effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:39.243 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:39.243 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 10 librados: watch_flush enter 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 10 librados: watch_flush exit 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 20 client.4752.objecter shutdown clearing up homeless session... 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 10 client.4752.objecter successfully canceled tick 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 msgr2=0x7f687807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 0x7f687807b340 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6898009c40 tx=0x7f6898009290 comp rx=0 tx=0).stop 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 10 monclient: shutdown 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 msgr2=0x7f68a80bcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68a4007e60 tx=0x7f68a400e4b0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 shutdown_connections 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6878078e80 0x7f687807b340 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f68a80c99a0 0x7f68a81eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f68a815a410 0x7f68a80bcaa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 --2- 10.20.193.18:0/3112445635 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f68a806ff20 0x7f68a8149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:39.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.242+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 >> 10.20.193.18:0/3112445635 conn(0x7f68a80cd780 msgr2=0x7f68a814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:39.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.243+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 shutdown_connections 2026-02-24T18:36:39.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.243+0000 7f68b0386640 1 -- 10.20.193.18:0/3112445635 wait complete. 2026-02-24T18:36:39.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:39.243+0000 7f68b0386640 1 librados: shutdown 2026-02-24T18:36:39.403 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:17.348 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.406 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:17.352 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.414 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:17.360 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.431 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:17.340 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.433 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:17.342 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.434 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:17.380 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.436 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:17.382 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.442 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:17.350 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:39.445 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:17.390 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:42.254 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 10 monclient: get_monmap_and_config 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 10 monclient: build_initial_monmap 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 10 monclient: monmap: 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:42.325075+0000 2026-02-24T18:36:42.325 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:42.325075+0000 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 1 Processor -- start 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 1 -- start start 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.324+0000 7f2177e58640 10 monclient: init 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient: _add_conns ranks=[2,5,3,1,0,4] 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f217006ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f21700c99a0 0x7f2170070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-c con 0x7f21700c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f217015a5e0 0x7f217014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-a con 0x7f217015a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21700c7e90 con 0x7f217006ff20 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:42.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2170159a40 con 0x7f217015a5e0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f21700c8170 con 0x7f21700c99a0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f2177e58640 10 monclient(hunting): authenticate will time out at 1546.006206s 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f217015a5e0 0x7f217014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f217015a5e0 0x7f217014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59344/0 (socket says 10.20.193.18:59344) 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 1 -- 10.20.193.18:0/155371154 learned_addr learned my addr 10.20.193.18:0/155371154 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 10 monclient(hunting): get_auth_request con 0x7f217015a5e0 auth_method 0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21763ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.325+0000 7f21753cc640 1 --2- 10.20.193.18:0/155371154 >> v2:10.20.193.110:3300/0 conn(0x7f21700c99a0 0x7f2170070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): get_auth_request con 0x7f21700c99a0 auth_method 0 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:42.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.326+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 10 monclient(hunting): handle_auth_done global_id 4755 payload 274 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 -- 10.20.193.18:0/155371154 >> v2:10.20.193.110:3300/0 conn(0x7f21700c99a0 msgr2=0x7f2170070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 --2- 10.20.193.18:0/155371154 >> v2:10.20.193.110:3300/0 conn(0x7f21700c99a0 0x7f2170070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 -- 10.20.193.18:0/155371154 >> v1:10.20.193.110:6789/0 conn(0x7f217006ff20 legacy=0x7f2170159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 10 monclient: _finish_hunting 0 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 monclient: found mon.noname-a 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 -- 10.20.193.18:0/155371154 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700c8450 con 0x7f217015a5e0 2026-02-24T18:36:42.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.327+0000 7f21763ce640 1 --2- 10.20.193.18:0/155371154 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f217015a5e0 0x7f217014fe10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f216c009e50 tx=0x7f216c02f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749e2d16a6cd4bde server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 1 -- 10.20.193.18:0/155371154 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c02fd60 con 0x7f217015a5e0 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: dump: 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: _finish_auth 0 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: _check_auth_tickets 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:42.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 1 -- 10.20.193.18:0/155371154 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216c0373f0 con 0x7f217015a5e0 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2177e58640 5 monclient: authenticate success, global_id 4755 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2177e58640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 1 -- 10.20.193.18:0/155371154 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c0377c0 con 0x7f217015a5e0 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2174bcb640 10 monclient: dump: 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:42.332 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2177e58640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.329+0000 7f2177e58640 4 set_mon_vals no callback set 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 10 monclient: shutdown 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 -- 10.20.193.18:0/155371154 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f217015a5e0 msgr2=0x7f217014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 --2- 10.20.193.18:0/155371154 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f217015a5e0 0x7f217014fe10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f216c009e50 tx=0x7f216c02f2a0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 -- 10.20.193.18:0/155371154 shutdown_connections 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 --2- 10.20.193.18:0/155371154 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f217015a5e0 0x7f217014fe10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 --2- 10.20.193.18:0/155371154 >> v2:10.20.193.110:3300/0 conn(0x7f21700c99a0 0x7f2170070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 -- 10.20.193.18:0/155371154 >> 10.20.193.18:0/155371154 conn(0x7f21700cd780 msgr2=0x7f21700cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:42.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 -- 10.20.193.18:0/155371154 shutdown_connections 2026-02-24T18:36:42.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 1 -- 10.20.193.18:0/155371154 wait complete. 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 10 monclient: build_initial_monmap 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.330+0000 7f2177e58640 10 monclient: monmap: 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.339 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 librados: starting msgr at 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 librados: starting objecter 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 Processor -- start 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 -- start start 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 librados: setting wanted keys 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 librados: calling monclient init 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient: init 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:42.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f217006ff20 0x7f21701f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-c con 0x7f217006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-a con 0x7f21700c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f217015a5e0 0x7f21701eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): picked mon.noname-b con 0x7f217015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2170150350 con 0x7f21700c99a0 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21700c8a10 con 0x7f217015a5e0 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:42.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21701504d0 con 0x7f217006ff20 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f2177e58640 10 monclient(hunting): authenticate will time out at 1546.012612s 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59352/0 (socket says 10.20.193.18:59352) 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 1 -- 10.20.193.18:0/262806484 learned_addr learned my addr 10.20.193.18:0/262806484 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 10 monclient(hunting): get_auth_request con 0x7f21700c99a0 auth_method 0 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.331+0000 7f21753cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f217006ff20 0x7f21701f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): get_auth_request con 0x7f217006ff20 auth_method 0 2026-02-24T18:36:42.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21753cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f217015a5e0 0x7f21701eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f2175bcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): get_auth_request con 0x7f217015a5e0 auth_method 0 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:42.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.332+0000 7f21763ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 10 monclient(hunting): handle_auth_done global_id 4758 payload 824 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 -- 10.20.193.18:0/262806484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f217006ff20 msgr2=0x7f21701f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f217006ff20 0x7f21701f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 -- 10.20.193.18:0/262806484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f217015a5e0 msgr2=0x7f21701eb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f217015a5e0 0x7f21701eb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 10 monclient: _finish_hunting 0 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 monclient: found mon.noname-a 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:42.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.333+0000 7f21753cc640 1 -- 10.20.193.18:0/262806484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700bf150 con 0x7f21700c99a0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.334+0000 7f21753cc640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f216400ec10 tx=0x7f216400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.334+0000 7f215e7fc640 10 client.?.objecter ms_handle_connect 0x7f21700c99a0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.334+0000 7f215e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164017070 con 0x7f21700c99a0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: dump: 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.345 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 5 monclient: authenticate success, global_id 4758 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216400cc40 con 0x7f21700c99a0 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:42.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164013470 con 0x7f21700c99a0 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215e7fc640 10 monclient: dump: 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:42.347 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 monclient: _renew_subs 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f215ffff640 4 set_mon_vals no callback set 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21700bf430 con 0x7f21700c99a0 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 client.4758.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 monclient: _renew_subs 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21700bf9f0 con 0x7f21700c99a0 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 1 librados: init done 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.336+0000 7f2177e58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:42.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.337+0000 7f2177e58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:42.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.337+0000 7f2177e58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.337+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2170150860 con 0x7f21700c99a0 2026-02-24T18:36:42.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.348+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2164013630 con 0x7f21700c99a0 2026-02-24T18:36:42.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.348+0000 7f215e7fc640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 0x7f214007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:42.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.349+0000 7f2175bcd640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 0x7f214007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:42.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.349+0000 7f2175bcd640 10 monclient: get_auth_request con 0x7f2140078e50 auth_method 0 2026-02-24T18:36:42.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.349+0000 7f2175bcd640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 0x7f214007b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f21600051f0 tx=0x7f2160007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:42.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.349+0000 7f215e7fc640 10 client.4758.objecter ms_handle_connect 0x7f2140078e50 2026-02-24T18:36:42.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f216405e510 con 0x7f21700c99a0 2026-02-24T18:36:42.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 10 client.4758.objecter ms_dispatch2 0x7f2170000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:42.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 3 client.4758.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:42.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 3 client.4758.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:42.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 20 client.4758.objecter dump_active .. 0 homeless 2026-02-24T18:36:42.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f21640624d0 con 0x7f21700c99a0 2026-02-24T18:36:42.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:42.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.450+0000 7f215e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:42.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.549+0000 7f2177e58640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:42.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.549+0000 7f2177e58640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:42.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.549+0000 7f2177e58640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:42.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.549+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2170158110 con 0x7f21700c99a0 2026-02-24T18:36:42.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.552+0000 7f215e7fc640 1 -- 10.20.193.18:0/262806484 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f21640667d0 con 0x7f21700c99a0 2026-02-24T18:36:42.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.552+0000 7f215e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:42.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.552+0000 7f215e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:42.554 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:42.554 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:42.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 10 librados: watch_flush enter 2026-02-24T18:36:42.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 10 librados: watch_flush exit 2026-02-24T18:36:42.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 20 client.4758.objecter shutdown clearing up homeless session... 2026-02-24T18:36:42.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 10 client.4758.objecter successfully canceled tick 2026-02-24T18:36:42.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 msgr2=0x7f214007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 0x7f214007b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f21600051f0 tx=0x7f2160007f30 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 10 monclient: shutdown 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 msgr2=0x7f21700c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f216400ec10 tx=0x7f216400c520 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 shutdown_connections 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2140078e50 0x7f214007b310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f217015a5e0 0x7f21701eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f21700c99a0 0x7f21700c04d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 --2- 10.20.193.18:0/262806484 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f217006ff20 0x7f21701f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:42.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 >> 10.20.193.18:0/262806484 conn(0x7f21700cd780 msgr2=0x7f217014e500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:42.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 shutdown_connections 2026-02-24T18:36:42.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 -- 10.20.193.18:0/262806484 wait complete. 2026-02-24T18:36:42.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:42.553+0000 7f2177e58640 1 librados: shutdown 2026-02-24T18:36:45.566 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:45.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.635+0000 7f5387bc3640 10 monclient: get_monmap_and_config 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.635+0000 7f5387bc3640 10 monclient: build_initial_monmap 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.635+0000 7f5387bc3640 10 monclient: monmap: 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:45.636568+0000 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:45.636568+0000 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.638 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 Processor -- start 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 -- start start 2026-02-24T18:36:45.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient: init 2026-02-24T18:36:45.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:45.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient: _add_conns ranks=[3,1,0,2,4,5] 2026-02-24T18:36:45.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f538006ff20 0x7f5380070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:45.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-a con 0x7f538006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f53800c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f538015a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53800c7e90 con 0x7f538015a5e0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53800c8170 con 0x7f53800c99a0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5380159a40 con 0x7f538006ff20 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.636+0000 7f5387bc3640 10 monclient(hunting): authenticate will time out at 1549.317720s 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f538006ff20 0x7f5380070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385137640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f53800c99a0 0x7f5380159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:44122/0 (socket says 10.20.193.18:44122) 2026-02-24T18:36:45.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385137640 1 -- 10.20.193.18:0/671542183 learned_addr learned my addr 10.20.193.18:0/671542183 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): get_auth_request con 0x7f538006ff20 auth_method 0 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.637+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 1 -- 10.20.193.18:0/671542183 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604949166 0 0) 0x7f53800c8170 con 0x7f53800c99a0 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 10 monclient(hunting): _negotiate 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:45.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 10 monclient(hunting): my global_id is 4738 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5384936640 1 -- 10.20.193.18:0/671542183 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5368003620 con 0x7f53800c99a0 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 10 monclient(hunting): handle_auth_done global_id 4761 payload 274 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 1 -- 10.20.193.18:0/671542183 >> v1:10.20.193.32:6789/0 conn(0x7f53800c99a0 legacy=0x7f5380159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 1 -- 10.20.193.18:0/671542183 >> v1:10.20.193.18:6789/0 conn(0x7f538015a5e0 legacy=0x7f538014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 10 monclient: _finish_hunting 0 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 1 monclient: found mon.noname-a 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.638+0000 7f5385938640 1 -- 10.20.193.18:0/671542183 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53800c8450 con 0x7f538006ff20 2026-02-24T18:36:45.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.639+0000 7f5385938640 1 --2- 10.20.193.18:0/671542183 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f5380070300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53700098d0 tx=0x7f537002ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd48f2e30bb9dc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 1 -- 10.20.193.18:0/671542183 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537003c070 con 0x7f538006ff20 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: dump: 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:45.652 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: _finish_auth 0 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: _check_auth_tickets 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 1 -- 10.20.193.18:0/671542183 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f537002fb00 con 0x7f538006ff20 2026-02-24T18:36:45.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 5 monclient: authenticate success, global_id 4761 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 1 -- 10.20.193.18:0/671542183 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370037440 con 0x7f538006ff20 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 4 set_mon_vals no callback set 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5384936640 10 monclient: dump: 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.654 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 10 monclient: shutdown 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 -- 10.20.193.18:0/671542183 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 msgr2=0x7f5380070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 --2- 10.20.193.18:0/671542183 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f5380070300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53700098d0 tx=0x7f537002ee80 comp rx=0 tx=0).stop 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 -- 10.20.193.18:0/671542183 shutdown_connections 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 --2- 10.20.193.18:0/671542183 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f5380070300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 -- 10.20.193.18:0/671542183 >> 10.20.193.18:0/671542183 conn(0x7f53800cd780 msgr2=0x7f53800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 -- 10.20.193.18:0/671542183 shutdown_connections 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 1 -- 10.20.193.18:0/671542183 wait complete. 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 10 monclient: build_initial_monmap 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.642+0000 7f5387bc3640 10 monclient: monmap: 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:45.656 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 librados: starting msgr at 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 librados: starting objecter 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 Processor -- start 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 -- start start 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 librados: setting wanted keys 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 librados: calling monclient init 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient: init 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:45.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-a con 0x7f538006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53800c99a0 0x7f538006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-c con 0x7f53800c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f538015a5e0 0x7f53801f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): picked mon.noname-b con 0x7f538015a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5380150520 con 0x7f538006ff20 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53800c8a10 con 0x7f538015a5e0 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53801506a0 con 0x7f53800c99a0 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5387bc3640 10 monclient(hunting): authenticate will time out at 1549.324652s 2026-02-24T18:36:45.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.643+0000 7f5385938640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59374/0 (socket says 10.20.193.18:59374) 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 1 -- 10.20.193.18:0/4240843099 learned_addr learned my addr 10.20.193.18:0/4240843099 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): get_auth_request con 0x7f538006ff20 auth_method 0 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f538015a5e0 0x7f53801f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53800c99a0 0x7f538006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:45.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): get_auth_request con 0x7f538015a5e0 auth_method 0 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): get_auth_request con 0x7f53800c99a0 auth_method 0 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5386139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:45.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.644+0000 7f5385137640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 10 monclient(hunting): handle_auth_done global_id 4764 payload 824 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 -- 10.20.193.18:0/4240843099 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53800c99a0 msgr2=0x7f538006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53800c99a0 0x7f538006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 -- 10.20.193.18:0/4240843099 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f538015a5e0 msgr2=0x7f53801f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f538015a5e0 0x7f53801f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 10 monclient: _finish_hunting 0 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 monclient: found mon.noname-a 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.645+0000 7f5385938640 1 -- 10.20.193.18:0/4240843099 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53801f3ef0 con 0x7f538006ff20 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.646+0000 7f5385938640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f537003a040 tx=0x7f53700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:45.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.646+0000 7f536e7fc640 10 client.?.objecter ms_handle_connect 0x7f538006ff20 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.646+0000 7f536e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370046020 con 0x7f538006ff20 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: dump: 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.662 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5370042620 con 0x7f538006ff20 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 5 monclient: authenticate success, global_id 4764 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _renew_subs 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538006e030 con 0x7f538006ff20 2026-02-24T18:36:45.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537003c040 con 0x7f538006ff20 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536ffff640 4 set_mon_vals no callback set 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f536e7fc640 10 monclient: dump: 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:45.664 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 client.4764.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _renew_subs 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538006e5f0 con 0x7f538006ff20 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 1 librados: init done 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.648+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5380070300 con 0x7f538006ff20 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.660+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f537003f030 con 0x7f538006ff20 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.660+0000 7f536e7fc640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 0x7f535007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.660+0000 7f5385137640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 0x7f535007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.660+0000 7f5385137640 10 monclient: get_auth_request con 0x7f5350078e80 auth_method 0 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 10 client.4764.objecter ms_handle_connect 0x7f5350078e80 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f5385137640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 0x7f535007b340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5374007b60 tx=0x7f5374007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f537002f9c0 con 0x7f538006ff20 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 10 client.4764.objecter ms_dispatch2 0x7f5380000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 3 client.4764.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:45.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 3 client.4764.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:45.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.661+0000 7f536e7fc640 20 client.4764.objecter dump_active .. 0 homeless 2026-02-24T18:36:45.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.759+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5370050030 con 0x7f538006ff20 2026-02-24T18:36:45.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.759+0000 7f536e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:45.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.759+0000 7f536e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:45.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.859+0000 7f5387bc3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:45.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.859+0000 7f5387bc3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:45.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.859+0000 7f5387bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:45.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.859+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f538006dd60 con 0x7f538006ff20 2026-02-24T18:36:45.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.862+0000 7f536e7fc640 1 -- 10.20.193.18:0/4240843099 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5370087140 con 0x7f538006ff20 2026-02-24T18:36:45.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.862+0000 7f536e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:45.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.862+0000 7f536e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:45.863 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:45.863 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:45.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 10 librados: watch_flush enter 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 10 librados: watch_flush exit 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 20 client.4764.objecter shutdown clearing up homeless session... 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 10 client.4764.objecter successfully canceled tick 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 msgr2=0x7f535007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 0x7f535007b340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5374007b60 tx=0x7f5374007ab0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 10 monclient: shutdown 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 msgr2=0x7f538006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f537003a040 tx=0x7f53700047c0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 shutdown_connections 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5350078e80 0x7f535007b340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f538015a5e0 0x7f53801f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f53800c99a0 0x7f538006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 --2- 10.20.193.18:0/4240843099 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f538006ff20 0x7f538006edf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 >> 10.20.193.18:0/4240843099 conn(0x7f53800cd780 msgr2=0x7f5380072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 shutdown_connections 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 -- 10.20.193.18:0/4240843099 wait complete. 2026-02-24T18:36:45.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:45.863+0000 7f5387bc3640 1 librados: shutdown 2026-02-24T18:36:48.874 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.943+0000 7f6190258640 10 monclient: get_monmap_and_config 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.943+0000 7f6190258640 10 monclient: build_initial_monmap 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.943+0000 7f6190258640 10 monclient: monmap: 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:48.944803+0000 2026-02-24T18:36:48.946 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:48.944803+0000 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.944+0000 7f6190258640 1 Processor -- start 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.944+0000 7f6190258640 1 -- start start 2026-02-24T18:36:48.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.944+0000 7f6190258640 10 monclient: init 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient: _add_conns ranks=[2,4,5,0,3,1] 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f618806ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f61880c99a0 0x7f6188070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): picked mon.noname-b con 0x7f61880c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f618815a5e0 0x7f618814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): picked mon.noname-c con 0x7f618815a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61880c7e90 con 0x7f618806ff20 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:48.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6188159a40 con 0x7f61880c99a0 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f61880c8170 con 0x7f618815a5e0 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f6190258640 10 monclient(hunting): authenticate will time out at 1552.626110s 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618d7cc640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f61880c99a0 0x7f6188070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618dfcd640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f618806ff20 0x7f6188159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:46112/0 (socket says 10.20.193.18:46112) 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618dfcd640 1 -- 10.20.193.18:0/3989635542 learned_addr learned my addr 10.20.193.18:0/3989635542 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618d7cc640 10 monclient(hunting): get_auth_request con 0x7f61880c99a0 auth_method 0 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618d7cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618d7cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.945+0000 7f618d7cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:48.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618e7ce640 1 --2- 10.20.193.18:0/3989635542 >> v2:10.20.193.110:3300/0 conn(0x7f618815a5e0 0x7f618814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618e7ce640 10 monclient(hunting): get_auth_request con 0x7f618815a5e0 auth_method 0 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618e7ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618e7ce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.946+0000 7f618e7ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 10 monclient(hunting): handle_auth_done global_id 4744 payload 274 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 1 -- 10.20.193.18:0/3989635542 >> v2:10.20.193.110:3300/0 conn(0x7f618815a5e0 msgr2=0x7f618814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:48.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 1 --2- 10.20.193.18:0/3989635542 >> v2:10.20.193.110:3300/0 conn(0x7f618815a5e0 0x7f618814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 1 -- 10.20.193.18:0/3989635542 >> v1:10.20.193.110:6789/0 conn(0x7f618806ff20 legacy=0x7f6188159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 10 monclient: _finish_hunting 0 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 1 monclient: found mon.noname-b 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.947+0000 7f618d7cc640 1 -- 10.20.193.18:0/3989635542 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880c8450 con 0x7f61880c99a0 2026-02-24T18:36:48.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.948+0000 7f618d7cc640 1 --2- 10.20.193.18:0/3989635542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f61880c99a0 0x7f6188070300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098a0 tx=0x7f617c02ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b20313850a4755 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 1 -- 10.20.193.18:0/3989635542 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c03c070 con 0x7f61880c99a0 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: dump: 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:48.960 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: _finish_auth 0 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: _check_auth_tickets 2026-02-24T18:36:48.961 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 1 -- 10.20.193.18:0/3989635542 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c02fae0 con 0x7f61880c99a0 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 5 monclient: authenticate success, global_id 4744 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 1 -- 10.20.193.18:0/3989635542 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c037440 con 0x7f61880c99a0 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 4 set_mon_vals no callback set 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f618cfcb640 10 monclient: dump: 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:48.962 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 10 monclient: shutdown 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 -- 10.20.193.18:0/3989635542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f61880c99a0 msgr2=0x7f6188070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 --2- 10.20.193.18:0/3989635542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f61880c99a0 0x7f6188070300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098a0 tx=0x7f617c02ee10 comp rx=0 tx=0).stop 2026-02-24T18:36:48.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 -- 10.20.193.18:0/3989635542 shutdown_connections 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 --2- 10.20.193.18:0/3989635542 >> v2:10.20.193.110:3300/0 conn(0x7f618815a5e0 0x7f618814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 --2- 10.20.193.18:0/3989635542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f61880c99a0 0x7f6188070300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 -- 10.20.193.18:0/3989635542 >> 10.20.193.18:0/3989635542 conn(0x7f61880cd780 msgr2=0x7f61880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 -- 10.20.193.18:0/3989635542 shutdown_connections 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.950+0000 7f6190258640 1 -- 10.20.193.18:0/3989635542 wait complete. 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 10 monclient: build_initial_monmap 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 10 monclient: monmap: 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:48.964 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 librados: starting msgr at 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 librados: starting objecter 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 Processor -- start 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 -- start start 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 librados: setting wanted keys 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 librados: calling monclient init 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 10 monclient: init 2026-02-24T18:36:48.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.951+0000 7f6190258640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): picked mon.noname-c con 0x7f618806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f61880c99a0 0x7f61881f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): picked mon.noname-a con 0x7f61880c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f618815a5e0 0x7f61881f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): picked mon.noname-b con 0x7f618815a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6188150350 con 0x7f61880c99a0 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61880c8a10 con 0x7f618815a5e0 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61881504d0 con 0x7f618806ff20 2026-02-24T18:36:48.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f6190258640 10 monclient(hunting): authenticate will time out at 1552.633030s 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f618815a5e0 0x7f61881f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f61880c99a0 0x7f61881f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:40346/0 (socket says 10.20.193.18:40346) 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 1 -- 10.20.193.18:0/2887351789 learned_addr learned my addr 10.20.193.18:0/2887351789 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): get_auth_request con 0x7f618806ff20 auth_method 0 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): get_auth_request con 0x7f618815a5e0 auth_method 0 2026-02-24T18:36:48.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): get_auth_request con 0x7f61880c99a0 auth_method 0 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618dfcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618e7ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:48.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.952+0000 7f618d7cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 10 monclient(hunting): handle_auth_done global_id 4730 payload 824 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 -- 10.20.193.18:0/2887351789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f618815a5e0 msgr2=0x7f61881f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f618815a5e0 0x7f61881f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 -- 10.20.193.18:0/2887351789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f61880c99a0 msgr2=0x7f61881f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f61880c99a0 0x7f61881f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 10 monclient: _finish_hunting 0 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 monclient: found mon.noname-c 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.954+0000 7f618dfcd640 1 -- 10.20.193.18:0/2887351789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6188156780 con 0x7f618806ff20 2026-02-24T18:36:48.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.955+0000 7f618dfcd640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f617800c760 tx=0x7f617800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.955+0000 7f61767fc640 10 client.?.objecter ms_handle_connect 0x7f618806ff20 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.955+0000 7f61767fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178017070 con 0x7f618806ff20 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: dump: 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.970 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178005b90 con 0x7f618806ff20 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 5 monclient: authenticate success, global_id 4730 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 10 monclient: _renew_subs 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6188156a60 con 0x7f618806ff20 2026-02-24T18:36:48.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178007070 con 0x7f618806ff20 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6177fff640 4 set_mon_vals no callback set 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f61767fc640 10 monclient: dump: 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:48.972 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 10 client.4730.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 10 monclient: _renew_subs 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6188157020 con 0x7f618806ff20 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.956+0000 7f6190258640 1 librados: init done 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.957+0000 7f6190258640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.957+0000 7f6190258640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.957+0000 7f6190258640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:48.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.957+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6188157260 con 0x7f618806ff20 2026-02-24T18:36:48.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.968+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6178007210 con 0x7f618806ff20 2026-02-24T18:36:48.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.969+0000 7f61767fc640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 0x7f615807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:48.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.969+0000 7f618d7cc640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 0x7f615807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:48.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.969+0000 7f618d7cc640 10 monclient: get_auth_request con 0x7f6158078e50 auth_method 0 2026-02-24T18:36:48.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.970+0000 7f618d7cc640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 0x7f615807b310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c039000 tx=0x7f617c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:48.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:48.970+0000 7f61767fc640 10 client.4730.objecter ms_handle_connect 0x7f6158078e50 2026-02-24T18:36:49.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.062+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f617805d4b0 con 0x7f618806ff20 2026-02-24T18:36:49.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.062+0000 7f61767fc640 10 client.4730.objecter ms_dispatch2 0x7f6188000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:49.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.062+0000 7f61767fc640 3 client.4730.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:49.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.062+0000 7f61767fc640 3 client.4730.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:49.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.062+0000 7f61767fc640 20 client.4730.objecter dump_active .. 0 homeless 2026-02-24T18:36:49.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.070+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6188157260 con 0x7f618806ff20 2026-02-24T18:36:49.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.070+0000 7f61767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:49.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.070+0000 7f61767fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:49.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.169+0000 7f6190258640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:49.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.169+0000 7f6190258640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:49.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.169+0000 7f6190258640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:36:49.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.169+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6188150860 con 0x7f618806ff20 2026-02-24T18:36:49.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f61767fc640 1 -- 10.20.193.18:0/2887351789 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6178021020 con 0x7f618806ff20 2026-02-24T18:36:49.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f61767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:49.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f61767fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:49.174 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:49.174 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:49.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 10 librados: watch_flush enter 2026-02-24T18:36:49.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 10 librados: watch_flush exit 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 20 client.4730.objecter shutdown clearing up homeless session... 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 10 client.4730.objecter successfully canceled tick 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 msgr2=0x7f615807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 0x7f615807b310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c039000 tx=0x7f617c033000 comp rx=0 tx=0).stop 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 10 monclient: shutdown 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 msgr2=0x7f618806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f617800c760 tx=0x7f617800cc30 comp rx=0 tx=0).stop 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 shutdown_connections 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6158078e50 0x7f615807b310 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f618815a5e0 0x7f61881f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f61880c99a0 0x7f61881f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 --2- 10.20.193.18:0/2887351789 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f618806ff20 0x7f618806e960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 >> 10.20.193.18:0/2887351789 conn(0x7f61880cd780 msgr2=0x7f61880c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 shutdown_connections 2026-02-24T18:36:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 -- 10.20.193.18:0/2887351789 wait complete. 2026-02-24T18:36:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:49.172+0000 7f6190258640 1 librados: shutdown 2026-02-24T18:36:52.184 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 10 monclient: get_monmap_and_config 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 10 monclient: build_initial_monmap 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 10 monclient: monmap: 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:52.251925+0000 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:52.251925+0000 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:52.252 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:52.253 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:52.253 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:52.253 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:52.253 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:52.253 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 1 Processor -- start 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 1 -- start start 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.251+0000 7f1d77b55640 10 monclient: init 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient: _add_conns ranks=[2,0,4,3,1,5] 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d701527f0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1d70073530 addr v1:10.20.193.18:6789/0 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f1d7015fad0 0x7f1d700ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-b con 0x7f1d7015fad0 addr v2:10.20.193.32:3300/0 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d700739a0 con 0x7f1d70073530 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7015d9d0 con 0x7f1d701527f0 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1d70074b10 con 0x7f1d7015fad0 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d77b55640 10 monclient(hunting): authenticate will time out at 1555.933177s 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d750c9640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1d70073530 0x7f1d7015f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:52858/0 (socket says 10.20.193.18:52858) 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d750c9640 1 -- 10.20.193.18:0/3138927905 learned_addr learned my addr 10.20.193.18:0/3138927905 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:52.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d760cb640 1 --2- 10.20.193.18:0/3138927905 >> v2:10.20.193.32:3300/0 conn(0x7f1d7015fad0 0x7f1d700ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d760cb640 10 monclient(hunting): get_auth_request con 0x7f1d7015fad0 auth_method 0 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d760cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d760cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.252+0000 7f1d760cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844573882 0 0) 0x7f1d64002920 con 0x7f1d70073530 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _negotiate 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): my global_id is 4770 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d58003620 con 0x7f1d70073530 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938106845 0 0) 0x7f1d7015d9d0 con 0x7f1d701527f0 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _negotiate 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 10 monclient(hunting): my global_id is 4733 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.253+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d64002920 con 0x7f1d701527f0 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 10 monclient(hunting): handle_auth_done global_id 4750 payload 274 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 1 -- 10.20.193.18:0/3138927905 >> v1:10.20.193.110:6789/0 conn(0x7f1d701527f0 legacy=0x7f1d7015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 1 -- 10.20.193.18:0/3138927905 >> v1:10.20.193.18:6789/0 conn(0x7f1d70073530 legacy=0x7f1d7015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 10 monclient: _finish_hunting 0 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 1 monclient: found mon.noname-b 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:36:52.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.254+0000 7f1d760cb640 1 -- 10.20.193.18:0/3138927905 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d70073c20 con 0x7f1d7015fad0 2026-02-24T18:36:52.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.255+0000 7f1d760cb640 1 --2- 10.20.193.18:0/3138927905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700ccf70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009d90 tx=0x7f1d6c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3efd8dbff4d4fd69 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c03c070 con 0x7f1d7015fad0 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 10 monclient: dump: 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 10 monclient: _finish_auth 0 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 10 monclient: _check_auth_tickets 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d748c8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d77b55640 5 monclient: authenticate success, global_id 4750 2026-02-24T18:36:52.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.256+0000 7f1d77b55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6c037440 con 0x7f1d7015fad0 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 1 -- 10.20.193.18:0/3138927905 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0377f0 con 0x7f1d7015fad0 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d748c8640 10 monclient: dump: 2026-02-24T18:36:52.259 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d77b55640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.257+0000 7f1d77b55640 4 set_mon_vals no callback set 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 10 monclient: shutdown 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- 10.20.193.18:0/3138927905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 msgr2=0x7f1d700ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 --2- 10.20.193.18:0/3138927905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700ccf70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009d90 tx=0x7f1d6c02f230 comp rx=0 tx=0).stop 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- 10.20.193.18:0/3138927905 shutdown_connections 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 --2- 10.20.193.18:0/3138927905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700ccf70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- 10.20.193.18:0/3138927905 >> 10.20.193.18:0/3138927905 conn(0x7f1d700ce730 msgr2=0x7f1d700ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- 10.20.193.18:0/3138927905 shutdown_connections 2026-02-24T18:36:52.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- 10.20.193.18:0/3138927905 wait complete. 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 10 monclient: build_initial_monmap 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 10 monclient: monmap: 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 librados: starting msgr at 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 librados: starting objecter 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 Processor -- start 2026-02-24T18:36:52.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 -- start start 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 librados: setting wanted keys 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 1 librados: calling monclient init 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.258+0000 7f1d77b55640 10 monclient: init 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d70073530 0x7f1d700c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-c con 0x7f1d70073530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-a con 0x7f1d701527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): picked mon.noname-b con 0x7f1d7015fad0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700744c0 con 0x7f1d701527f0 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700741e0 con 0x7f1d7015fad0 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700747a0 con 0x7f1d70073530 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d77b55640 10 monclient(hunting): authenticate will time out at 1555.940422s 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d760cb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53040/0 (socket says 10.20.193.18:53040) 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 1 -- 10.20.193.18:0/2762423444 learned_addr learned my addr 10.20.193.18:0/2762423444 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 10 monclient(hunting): get_auth_request con 0x7f1d701527f0 auth_method 0 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:52.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d750c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d758ca640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d70073530 0x7f1d700c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d760cb640 10 monclient(hunting): get_auth_request con 0x7f1d7015fad0 auth_method 0 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d760cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d760cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d760cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.259+0000 7f1d758ca640 10 monclient(hunting): get_auth_request con 0x7f1d70073530 auth_method 0 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d750c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d750c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d750c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d760cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:52.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.260+0000 7f1d758ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 10 monclient(hunting): handle_auth_done global_id 4773 payload 824 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 -- 10.20.193.18:0/2762423444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d70073530 msgr2=0x7f1d700c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d70073530 0x7f1d700c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 -- 10.20.193.18:0/2762423444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 msgr2=0x7f1d700c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 10 monclient: _finish_hunting 0 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 monclient: found mon.noname-a 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.261+0000 7f1d750c9640 1 -- 10.20.193.18:0/2762423444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700c6de0 con 0x7f1d701527f0 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.262+0000 7f1d750c9640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c7b0 tx=0x7f1d6400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.262+0000 7f1d5e7fc640 10 client.?.objecter ms_handle_connect 0x7f1d701527f0 2026-02-24T18:36:52.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.262+0000 7f1d5e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64017070 con 0x7f1d701527f0 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: dump: 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:52.266 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d64005e00 con 0x7f1d701527f0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640072b0 con 0x7f1d701527f0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5e7fc640 10 monclient: dump: 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 5 monclient: authenticate success, global_id 4773 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 10 monclient: _renew_subs 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d5ffff640 4 set_mon_vals no callback set 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d700c70c0 con 0x7f1d701527f0 2026-02-24T18:36:52.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 10 client.4773.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 10 monclient: _renew_subs 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d700c7680 con 0x7f1d701527f0 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.264+0000 7f1d77b55640 1 librados: init done 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.265+0000 7f1d77b55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.265+0000 7f1d77b55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.265+0000 7f1d77b55640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:52.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.265+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d70074fa0 con 0x7f1d701527f0 2026-02-24T18:36:52.278 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.276+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1d64007450 con 0x7f1d701527f0 2026-02-24T18:36:52.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.277+0000 7f1d5e7fc640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 0x7f1d4007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:52.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.277+0000 7f1d758ca640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 0x7f1d4007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:52.279 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.277+0000 7f1d758ca640 10 monclient: get_auth_request con 0x7f1d40078e00 auth_method 0 2026-02-24T18:36:52.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d5e7fc640 10 client.4773.objecter ms_handle_connect 0x7f1d40078e00 2026-02-24T18:36:52.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d758ca640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 0x7f1d4007b2c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d60007fd0 tx=0x7f1d60003db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:52.280 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f1d64007770 con 0x7f1d701527f0 2026-02-24T18:36:52.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d5e7fc640 10 client.4773.objecter ms_dispatch2 0x7f1d70000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:52.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d5e7fc640 3 client.4773.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:52.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.278+0000 7f1d5e7fc640 3 client.4773.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:52.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.279+0000 7f1d5e7fc640 20 client.4773.objecter dump_active .. 0 homeless 2026-02-24T18:36:52.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.376+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1d64021020 con 0x7f1d701527f0 2026-02-24T18:36:52.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.376+0000 7f1d5e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:52.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.376+0000 7f1d5e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:52.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.475+0000 7f1d77b55640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:52.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.475+0000 7f1d77b55640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:52.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.475+0000 7f1d77b55640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:52.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.475+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d7015d8c0 con 0x7f1d701527f0 2026-02-24T18:36:52.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.478+0000 7f1d5e7fc640 1 -- 10.20.193.18:0/2762423444 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1d700739a0 con 0x7f1d701527f0 2026-02-24T18:36:52.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.478+0000 7f1d5e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:52.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.478+0000 7f1d5e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:52.480 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:52.480 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:52.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 10 librados: watch_flush enter 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 10 librados: watch_flush exit 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 20 client.4773.objecter shutdown clearing up homeless session... 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 10 client.4773.objecter successfully canceled tick 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 msgr2=0x7f1d4007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 0x7f1d4007b2c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d60007fd0 tx=0x7f1d60003db0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 10 monclient: shutdown 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 msgr2=0x7f1d701f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c7b0 tx=0x7f1d6400cc80 comp rx=0 tx=0).stop 2026-02-24T18:36:52.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 shutdown_connections 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1d40078e00 0x7f1d4007b2c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1d7015fad0 0x7f1d700c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1d701527f0 0x7f1d701f4af0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 --2- 10.20.193.18:0/2762423444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1d70073530 0x7f1d700c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 >> 10.20.193.18:0/2762423444 conn(0x7f1d700ce730 msgr2=0x7f1d701516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 shutdown_connections 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 -- 10.20.193.18:0/2762423444 wait complete. 2026-02-24T18:36:52.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:52.479+0000 7f1d77b55640 1 librados: shutdown 2026-02-24T18:36:55.491 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: get_monmap_and_config 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: build_initial_monmap 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: monmap: 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:55.562 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:55.560874+0000 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:55.560874+0000 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 Processor -- start 2026-02-24T18:36:55.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 -- start start 2026-02-24T18:36:55.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: init 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient: _add_conns ranks=[1,3,2,5,0,4] 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fca2c06ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fca2c0c99a0 0x7fca2c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-a con 0x7fca2c0c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fca2c15a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:36:55.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2c0c7e90 con 0x7fca2c06ff20 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2c0c8170 con 0x7fca2c15a5e0 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c159a40 con 0x7fca2c0c99a0 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.560+0000 7fca33ccd640 10 monclient(hunting): authenticate will time out at 1559.242009s 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fca2c0c99a0 0x7fca2c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31a42640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fca2c06ff20 0x7fca2c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:49986/0 (socket says 10.20.193.18:49986) 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31a42640 1 -- 10.20.193.18:0/1539151129 learned_addr learned my addr 10.20.193.18:0/1539151129 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): get_auth_request con 0x7fca2c0c99a0 auth_method 0 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:55.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.561+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 10 monclient(hunting): handle_auth_done global_id 4776 payload 274 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 1 -- 10.20.193.18:0/1539151129 >> v1:10.20.193.110:6789/0 conn(0x7fca2c15a5e0 legacy=0x7fca2c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 1 -- 10.20.193.18:0/1539151129 >> v1:10.20.193.32:6789/0 conn(0x7fca2c06ff20 legacy=0x7fca2c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 10 monclient: _finish_hunting 0 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 1 monclient: found mon.noname-a 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.562+0000 7fca31241640 1 -- 10.20.193.18:0/1539151129 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c0c8450 con 0x7fca2c0c99a0 2026-02-24T18:36:55.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.563+0000 7fca31241640 1 --2- 10.20.193.18:0/1539151129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c070300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca200098d0 tx=0x7fca2002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430c0002857a9398 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:55.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 1 -- 10.20.193.18:0/1539151129 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2003c070 con 0x7fca2c0c99a0 2026-02-24T18:36:55.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:55.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:55.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: dump: 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: _finish_auth 0 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: _check_auth_tickets 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 1 -- 10.20.193.18:0/1539151129 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2002fba0 con 0x7fca2c0c99a0 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 1 -- 10.20.193.18:0/1539151129 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20037440 con 0x7fca2c0c99a0 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca30a40640 10 monclient: dump: 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:55.581 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 5 monclient: authenticate success, global_id 4776 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:55.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 4 set_mon_vals no callback set 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 10 monclient: shutdown 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 1 -- 10.20.193.18:0/1539151129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 msgr2=0x7fca2c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 1 --2- 10.20.193.18:0/1539151129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c070300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca200098d0 tx=0x7fca2002eed0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 1 -- 10.20.193.18:0/1539151129 shutdown_connections 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.565+0000 7fca33ccd640 1 --2- 10.20.193.18:0/1539151129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c070300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 -- 10.20.193.18:0/1539151129 >> 10.20.193.18:0/1539151129 conn(0x7fca2c0cd780 msgr2=0x7fca2c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 -- 10.20.193.18:0/1539151129 shutdown_connections 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 -- 10.20.193.18:0/1539151129 wait complete. 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 10 monclient: build_initial_monmap 2026-02-24T18:36:55.583 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 10 monclient: monmap: 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 librados: starting msgr at 2026-02-24T18:36:55.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 librados: starting objecter 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 Processor -- start 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 -- start start 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 librados: setting wanted keys 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 1 librados: calling monclient init 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.566+0000 7fca33ccd640 10 monclient: init 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca2c06ff20 0x7fca2c14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-b con 0x7fca2c06ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-a con 0x7fca2c0c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:55.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca2c15a5e0 0x7fca2c1e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): picked mon.noname-c con 0x7fca2c15a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c150520 con 0x7fca2c0c99a0 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c0c8a10 con 0x7fca2c06ff20 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c1506a0 con 0x7fca2c15a5e0 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca33ccd640 10 monclient(hunting): authenticate will time out at 1559.248299s 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53048/0 (socket says 10.20.193.18:53048) 2026-02-24T18:36:55.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 1 -- 10.20.193.18:0/908891545 learned_addr learned my addr 10.20.193.18:0/908891545 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): get_auth_request con 0x7fca2c0c99a0 auth_method 0 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31a42640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca2c06ff20 0x7fca2c14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca32243640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca2c15a5e0 0x7fca2c1e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca31241640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca32243640 10 monclient(hunting): get_auth_request con 0x7fca2c15a5e0 auth_method 0 2026-02-24T18:36:55.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca32243640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca32243640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.567+0000 7fca32243640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): get_auth_request con 0x7fca2c06ff20 auth_method 0 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca32243640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca32243640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca32243640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.568+0000 7fca31a42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:55.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 10 monclient(hunting): handle_auth_done global_id 4779 payload 824 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 -- 10.20.193.18:0/908891545 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca2c15a5e0 msgr2=0x7fca2c1e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca2c15a5e0 0x7fca2c1e51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 -- 10.20.193.18:0/908891545 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca2c06ff20 msgr2=0x7fca2c14c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca2c06ff20 0x7fca2c14c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 10 monclient: _finish_hunting 0 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 monclient: found mon.noname-a 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.569+0000 7fca31241640 1 -- 10.20.193.18:0/908891545 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c1e5cc0 con 0x7fca2c0c99a0 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.570+0000 7fca31241640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca20016ea0 tx=0x7fca200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.570+0000 7fca1a7fc640 10 client.?.objecter ms_handle_connect 0x7fca2c0c99a0 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.570+0000 7fca1a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:55.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20046020 con 0x7fca2c0c99a0 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: dump: 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:55.590 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca20004230 con 0x7fca2c0c99a0 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2003c040 con 0x7fca2c0c99a0 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca33ccd640 5 monclient: authenticate success, global_id 4779 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca33ccd640 10 monclient: _renew_subs 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.571+0000 7fca1bfff640 4 set_mon_vals no callback set 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c1e5fa0 con 0x7fca2c0c99a0 2026-02-24T18:36:55.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca1a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca1a7fc640 10 monclient: dump: 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:55.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 client.4779.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 monclient: _renew_subs 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c1e6560 con 0x7fca2c0c99a0 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 1 librados: init done 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.572+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c070300 con 0x7fca2c0c99a0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.584+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fca20053070 con 0x7fca2c0c99a0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.584+0000 7fca1a7fc640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 0x7fc9fc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.584+0000 7fca31a42640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 0x7fc9fc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.584+0000 7fca31a42640 10 monclient: get_auth_request con 0x7fc9fc078e50 auth_method 0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.585+0000 7fca31a42640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 0x7fc9fc07b310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca1c002770 tx=0x7fca1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.585+0000 7fca1a7fc640 10 client.4779.objecter ms_handle_connect 0x7fc9fc078e50 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.586+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fca2002fb00 con 0x7fca2c0c99a0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.586+0000 7fca1a7fc640 10 client.4779.objecter ms_dispatch2 0x7fca2c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.586+0000 7fca1a7fc640 3 client.4779.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.586+0000 7fca1a7fc640 3 client.4779.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:55.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.586+0000 7fca1a7fc640 20 client.4779.objecter dump_active .. 0 homeless 2026-02-24T18:36:55.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.689+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fca20050020 con 0x7fca2c0c99a0 2026-02-24T18:36:55.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.689+0000 7fca1a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:55.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.689+0000 7fca1a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:55.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.790+0000 7fca33ccd640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:55.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.790+0000 7fca33ccd640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:55.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.790+0000 7fca33ccd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:55.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.790+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fca2c14f5f0 con 0x7fca2c0c99a0 2026-02-24T18:36:55.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.792+0000 7fca1a7fc640 1 -- 10.20.193.18:0/908891545 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fca200875e0 con 0x7fca2c0c99a0 2026-02-24T18:36:55.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.792+0000 7fca1a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:55.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.792+0000 7fca1a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:55.794 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:55.794 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 10 librados: watch_flush enter 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 10 librados: watch_flush exit 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 20 client.4779.objecter shutdown clearing up homeless session... 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 10 client.4779.objecter successfully canceled tick 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 msgr2=0x7fc9fc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 0x7fc9fc07b310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca1c002770 tx=0x7fca1c009040 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 10 monclient: shutdown 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 msgr2=0x7fca2c0c0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca20016ea0 tx=0x7fca200047c0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 shutdown_connections 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc9fc078e50 0x7fc9fc07b310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fca2c15a5e0 0x7fca2c1e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fca2c0c99a0 0x7fca2c0c0630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 --2- 10.20.193.18:0/908891545 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fca2c06ff20 0x7fca2c14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 >> 10.20.193.18:0/908891545 conn(0x7fca2c0cd780 msgr2=0x7fca2c14d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 shutdown_connections 2026-02-24T18:36:55.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.793+0000 7fca33ccd640 1 -- 10.20.193.18:0/908891545 wait complete. 2026-02-24T18:36:55.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:55.794+0000 7fca33ccd640 1 librados: shutdown 2026-02-24T18:36:58.806 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:58.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 10 monclient: get_monmap_and_config 2026-02-24T18:36:58.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 10 monclient: build_initial_monmap 2026-02-24T18:36:58.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 10 monclient: monmap: 2026-02-24T18:36:58.878 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:58.878 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:36:58.877301+0000 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:36:58.877301+0000 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 1 Processor -- start 2026-02-24T18:36:58.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 1 -- start start 2026-02-24T18:36:58.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.876+0000 7fb62cd44640 10 monclient: init 2026-02-24T18:36:58.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:58.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient: _add_conns ranks=[3,0,5,1,2,4] 2026-02-24T18:36:58.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb6281527d0 0x7fb62806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:58.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-a con 0x7fb6281527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:36:58.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb628152bb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fb6280c6190 0x7fb6280c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-c con 0x7fb6280c6190 addr v2:10.20.193.110:3300/0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb62806e300 con 0x7fb628152bb0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb62806f1f0 con 0x7fb6281527d0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb62806e5e0 con 0x7fb6280c6190 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb62cd44640 10 monclient(hunting): authenticate will time out at 1562.558446s 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb626575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb6281527d0 0x7fb62806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.877+0000 7fb626d76640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fb6280c6190 0x7fb6280c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:58.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb6281527d0 0x7fb62806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53064/0 (socket says 10.20.193.18:53064) 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 1 -- 10.20.193.18:0/590677574 learned_addr learned my addr 10.20.193.18:0/590677574 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): get_auth_request con 0x7fb6281527d0 auth_method 0 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): get_auth_request con 0x7fb6280c6190 auth_method 0 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:58.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.878+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 10 monclient(hunting): handle_auth_done global_id 4782 payload 274 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 1 -- 10.20.193.18:0/590677574 >> v2:10.20.193.110:3300/0 conn(0x7fb6280c6190 msgr2=0x7fb6280c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 1 --2- 10.20.193.18:0/590677574 >> v2:10.20.193.110:3300/0 conn(0x7fb6280c6190 0x7fb6280c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 1 -- 10.20.193.18:0/590677574 >> v1:10.20.193.18:6789/0 conn(0x7fb628152bb0 legacy=0x7fb6280c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 10 monclient: _finish_hunting 0 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 1 monclient: found mon.noname-a 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.879+0000 7fb626575640 1 -- 10.20.193.18:0/590677574 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62806e8c0 con 0x7fb6281527d0 2026-02-24T18:36:58.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.880+0000 7fb626575640 1 --2- 10.20.193.18:0/590677574 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb6281527d0 0x7fb62806da60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb628158940 tx=0x7fb61c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8b31132e5caf50 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:58.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 1 -- 10.20.193.18:0/590677574 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c03c070 con 0x7fb6281527d0 2026-02-24T18:36:58.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:58.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: dump: 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: _finish_auth 0 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: _check_auth_tickets 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 5 monclient: authenticate success, global_id 4782 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 1 -- 10.20.193.18:0/590677574 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb61c02fb40 con 0x7fb6281527d0 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 1 -- 10.20.193.18:0/590677574 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c037440 con 0x7fb6281527d0 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 10 monclient: get_monmap_and_config success 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 4 set_mon_vals no callback set 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:58.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb625573640 10 monclient: dump: 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 10 monclient: shutdown 2026-02-24T18:36:58.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 -- 10.20.193.18:0/590677574 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb6281527d0 msgr2=0x7fb62806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 --2- 10.20.193.18:0/590677574 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb6281527d0 0x7fb62806da60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb628158940 tx=0x7fb61c02eed0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 -- 10.20.193.18:0/590677574 shutdown_connections 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 --2- 10.20.193.18:0/590677574 >> v2:10.20.193.110:3300/0 conn(0x7fb6280c6190 0x7fb6280c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 --2- 10.20.193.18:0/590677574 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb6281527d0 0x7fb62806da60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 -- 10.20.193.18:0/590677574 >> 10.20.193.18:0/590677574 conn(0x7fb6280cd780 msgr2=0x7fb6280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 -- 10.20.193.18:0/590677574 shutdown_connections 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.882+0000 7fb62cd44640 1 -- 10.20.193.18:0/590677574 wait complete. 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 10 monclient: build_initial_monmap 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 10 monclient: monmap: 2026-02-24T18:36:58.895 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 librados: starting msgr at 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 librados: starting objecter 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 Processor -- start 2026-02-24T18:36:58.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 -- start start 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 librados: setting wanted keys 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 1 librados: calling monclient init 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.883+0000 7fb62cd44640 10 monclient: init 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient: _reopen_session rank -1 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb6280c6190 0x7fb628151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-c con 0x7fb6280c6190 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6281527d0 0x7fb6280732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-b con 0x7fb6281527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): picked mon.noname-a con 0x7fb628152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:36:58.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb62806dfa0 con 0x7fb628152bb0 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb62806ee80 con 0x7fb6281527d0 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): start opening mon connection 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6280c93a0 con 0x7fb6280c6190 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): _renew_subs 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb62cd44640 10 monclient(hunting): authenticate will time out at 1562.565114s 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:53078/0 (socket says 10.20.193.18:53078) 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 1 -- 10.20.193.18:0/2323617367 learned_addr learned my addr 10.20.193.18:0/2323617367 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): get_auth_request con 0x7fb628152bb0 auth_method 0 2026-02-24T18:36:58.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626575640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb6280c6190 0x7fb628151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626575640 10 monclient(hunting): get_auth_request con 0x7fb6280c6190 auth_method 0 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb625d74640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6281527d0 0x7fb6280732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:58.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.884+0000 7fb626d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb626575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): get_auth_request con 0x7fb6281527d0 auth_method 0 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.885+0000 7fb625d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 10 monclient(hunting): handle_auth_done global_id 4785 payload 824 2026-02-24T18:36:58.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 -- 10.20.193.18:0/2323617367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb6280c6190 msgr2=0x7fb628151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb6280c6190 0x7fb628151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 -- 10.20.193.18:0/2323617367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6281527d0 msgr2=0x7fb6280732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6281527d0 0x7fb6280732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 10 monclient: _finish_hunting 0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 monclient: found mon.noname-a 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.886+0000 7fb626d76640 1 -- 10.20.193.18:0/2323617367 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628154c70 con 0x7fb628152bb0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.887+0000 7fb626d76640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb618009dd0 tx=0x7fb61800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.887+0000 7fb60effd640 10 client.?.objecter ms_handle_connect 0x7fb628152bb0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.887+0000 7fb60effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618018020 con 0x7fb628152bb0 2026-02-24T18:36:58.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: dump: 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:58.902 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: _finish_auth 0 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: _check_auth_tickets 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 5 monclient: authenticate success, global_id 4785 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _renew_subs 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb628154f50 con 0x7fb628152bb0 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 client.4785.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _renew_subs 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb628072af0 con 0x7fb628152bb0 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 1 librados: init done 2026-02-24T18:36:58.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6280c9780 con 0x7fb628152bb0 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb618014480 con 0x7fb628152bb0 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618014830 con 0x7fb628152bb0 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb60effd640 10 monclient: dump: 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.904 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:36:58.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.888+0000 7fb624d72640 4 set_mon_vals no callback set 2026-02-24T18:36:58.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.997+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb6180149d0 con 0x7fb628152bb0 2026-02-24T18:36:58.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.997+0000 7fb60effd640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 0x7fb5f407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:36:59.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.997+0000 7fb626575640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 0x7fb5f407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:36:59.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.997+0000 7fb626575640 10 monclient: get_auth_request con 0x7fb5f4078e00 auth_method 0 2026-02-24T18:36:59.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.998+0000 7fb626575640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 0x7fb5f407b2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6280750e0 tx=0x7fb61c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:36:59.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:58.998+0000 7fb60effd640 10 client.4785.objecter ms_handle_connect 0x7fb5f4078e00 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fb618098a20 con 0x7fb628152bb0 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 10 client.4785.objecter ms_dispatch2 0x7fb628000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 3 client.4785.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 3 client.4785.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 20 client.4785.objecter dump_active .. 0 homeless 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb618022070 con 0x7fb628152bb0 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:36:59.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.006+0000 7fb60effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:36:59.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.107+0000 7fb62cd44640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:36:59.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.107+0000 7fb62cd44640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:59.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.107+0000 7fb62cd44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:36:59.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.107+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb6280c85b0 con 0x7fb628152bb0 2026-02-24T18:36:59.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.111+0000 7fb60effd640 1 -- 10.20.193.18:0/2323617367 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb618010040 con 0x7fb628152bb0 2026-02-24T18:36:59.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.111+0000 7fb60effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:36:59.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.111+0000 7fb60effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:36:59.113 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:36:59.113 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:36:59.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 10 librados: watch_flush enter 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 10 librados: watch_flush exit 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 20 client.4785.objecter shutdown clearing up homeless session... 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 10 client.4785.objecter successfully canceled tick 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 msgr2=0x7fb5f407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 0x7fb5f407b2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6280750e0 tx=0x7fb61c033000 comp rx=0 tx=0).stop 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 10 monclient: shutdown 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 msgr2=0x7fb628073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb618009dd0 tx=0x7fb61800c490 comp rx=0 tx=0).stop 2026-02-24T18:36:59.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 shutdown_connections 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb5f4078e00 0x7fb5f407b2c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb6281527d0 0x7fb6280732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb628152bb0 0x7fb628073830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 --2- 10.20.193.18:0/2323617367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb6280c6190 0x7fb628151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 >> 10.20.193.18:0/2323617367 conn(0x7fb6280cd780 msgr2=0x7fb6280ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 shutdown_connections 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 -- 10.20.193.18:0/2323617367 wait complete. 2026-02-24T18:36:59.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:36:59.112+0000 7fb62cd44640 1 librados: shutdown 2026-02-24T18:36:59.429 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:37.374 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.430 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:37.376 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.438 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:37.383 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.457 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:37.402 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.457 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:37.403 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.458 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:37.366 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.458 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:37.367 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.465 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:37.411 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:36:59.466 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:37.375 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:02.124 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:02.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.194+0000 7f82a79bc640 10 monclient: get_monmap_and_config 2026-02-24T18:37:02.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.194+0000 7f82a79bc640 10 monclient: build_initial_monmap 2026-02-24T18:37:02.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.194+0000 7f82a79bc640 10 monclient: monmap: 2026-02-24T18:37:02.197 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:02.196620+0000 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:02.196620+0000 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:02.198 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:02.199 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 Processor -- start 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 -- start start 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient: init 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient: _add_conns ranks=[5,0,3,2,1,4] 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f82a006ff20 0x7f82a0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-c con 0x7f82a006ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82a00c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f82a015a5e0 0x7f82a014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-a con 0x7f82a015a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a00c7e90 con 0x7f82a00c99a0 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f82a0159a40 con 0x7f82a015a5e0 2026-02-24T18:37:02.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f82a00c8170 con 0x7f82a006ff20 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.195+0000 7f82a79bc640 10 monclient(hunting): authenticate will time out at 1565.877802s 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f82a015a5e0 0x7f82a014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f82a006ff20 0x7f82a0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f82a015a5e0 0x7f82a014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55666/0 (socket says 10.20.193.18:55666) 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 1 -- 10.20.193.18:0/3927713632 learned_addr learned my addr 10.20.193.18:0/3927713632 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): get_auth_request con 0x7f82a015a5e0 auth_method 0 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): get_auth_request con 0x7f82a006ff20 auth_method 0 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.196+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 10 monclient(hunting): handle_auth_done global_id 4788 payload 274 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 1 -- 10.20.193.18:0/3927713632 >> v2:10.20.193.110:3300/0 conn(0x7f82a006ff20 msgr2=0x7f82a0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 1 --2- 10.20.193.18:0/3927713632 >> v2:10.20.193.110:3300/0 conn(0x7f82a006ff20 0x7f82a0070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 1 -- 10.20.193.18:0/3927713632 >> v1:10.20.193.18:6789/0 conn(0x7f82a00c99a0 legacy=0x7f82a0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 10 monclient: _finish_hunting 0 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 1 monclient: found mon.noname-a 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.197+0000 7f82a5f32640 1 -- 10.20.193.18:0/3927713632 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a00c8450 con 0x7f82a015a5e0 2026-02-24T18:37:02.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.198+0000 7f82a5f32640 1 --2- 10.20.193.18:0/3927713632 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a014fe10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c02f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769da6c1f5e93bfe server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:02.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 1 -- 10.20.193.18:0/3927713632 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c02fcc0 con 0x7f82a015a5e0 2026-02-24T18:37:02.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:02.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:02.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: dump: 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: _finish_auth 0 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 1 -- 10.20.193.18:0/3927713632 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c0373f0 con 0x7f82a015a5e0 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f82a79bc640 5 monclient: authenticate success, global_id 4788 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 1 -- 10.20.193.18:0/3927713632 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c02fe60 con 0x7f82a015a5e0 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f82a79bc640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f82a79bc640 4 set_mon_vals no callback set 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.200+0000 7f828ffff640 10 monclient: dump: 2026-02-24T18:37:02.207 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 10 monclient: shutdown 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- 10.20.193.18:0/3927713632 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 msgr2=0x7f82a014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 --2- 10.20.193.18:0/3927713632 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a014fe10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c02f2a0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- 10.20.193.18:0/3927713632 shutdown_connections 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 --2- 10.20.193.18:0/3927713632 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a014fe10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 --2- 10.20.193.18:0/3927713632 >> v2:10.20.193.110:3300/0 conn(0x7f82a006ff20 0x7f82a0070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- 10.20.193.18:0/3927713632 >> 10.20.193.18:0/3927713632 conn(0x7f82a00cd780 msgr2=0x7f82a00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- 10.20.193.18:0/3927713632 shutdown_connections 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- 10.20.193.18:0/3927713632 wait complete. 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 10 monclient: build_initial_monmap 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 10 monclient: monmap: 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:02.208 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 librados: starting msgr at 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 librados: starting objecter 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 Processor -- start 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 -- start start 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 librados: setting wanted keys 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 1 librados: calling monclient init 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.201+0000 7f82a79bc640 10 monclient: init 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f82a006ff20 0x7f82a00c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-b con 0x7f82a006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f82a00c99a0 0x7f82a00c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-c con 0x7f82a00c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): picked mon.noname-a con 0x7f82a015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0150350 con 0x7f82a015a5e0 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82a00c8a10 con 0x7f82a006ff20 2026-02-24T18:37:02.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01504d0 con 0x7f82a00c99a0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a79bc640 10 monclient(hunting): authenticate will time out at 1565.884285s 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55668/0 (socket says 10.20.193.18:55668) 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 1 -- 10.20.193.18:0/279963592 learned_addr learned my addr 10.20.193.18:0/279963592 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): get_auth_request con 0x7f82a015a5e0 auth_method 0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a4f30640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f82a00c99a0 0x7f82a00c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5731640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f82a006ff20 0x7f82a00c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.202+0000 7f82a5f32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): get_auth_request con 0x7f82a006ff20 auth_method 0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): get_auth_request con 0x7f82a00c99a0 auth_method 0 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:02.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a4f30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.203+0000 7f82a5731640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 10 monclient(hunting): handle_auth_done global_id 4791 payload 824 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 -- 10.20.193.18:0/279963592 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f82a00c99a0 msgr2=0x7f82a00c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f82a00c99a0 0x7f82a00c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 -- 10.20.193.18:0/279963592 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f82a006ff20 msgr2=0x7f82a00c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f82a006ff20 0x7f82a00c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 10 monclient: _finish_hunting 0 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 monclient: found mon.noname-a 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.204+0000 7f82a5f32640 1 -- 10.20.193.18:0/279963592 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a00c10d0 con 0x7f82a015a5e0 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.205+0000 7f82a5f32640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.205+0000 7f828dffb640 10 client.?.objecter ms_handle_connect 0x7f82a015a5e0 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.205+0000 7f828dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c046020 con 0x7f82a015a5e0 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: dump: 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:02.212 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c041450 con 0x7f82a015a5e0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 5 monclient: authenticate success, global_id 4791 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 10 monclient: _renew_subs 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a01e6ea0 con 0x7f82a015a5e0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c009510 con 0x7f82a015a5e0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828f7fe640 4 set_mon_vals no callback set 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f828dffb640 10 monclient: dump: 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:02.213 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 10 client.4791.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 10 monclient: _renew_subs 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a00c1620 con 0x7f82a015a5e0 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.206+0000 7f82a79bc640 1 librados: init done 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.207+0000 7f82a79bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.207+0000 7f82a79bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.207+0000 7f82a79bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.207+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a0150860 con 0x7f82a015a5e0 2026-02-24T18:37:02.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.219+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f829c041820 con 0x7f82a015a5e0 2026-02-24T18:37:02.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.219+0000 7f828dffb640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 0x7f827007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:02.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.219+0000 7f82a5731640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 0x7f827007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:02.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.219+0000 7f82a5731640 10 monclient: get_auth_request con 0x7f8270078e30 auth_method 0 2026-02-24T18:37:02.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.220+0000 7f82a5731640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 0x7f827007b2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8290009c40 tx=0x7f8290009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:02.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.220+0000 7f828dffb640 10 client.4791.objecter ms_handle_connect 0x7f8270078e30 2026-02-24T18:37:02.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f829c0838e0 con 0x7f82a015a5e0 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 10 client.4791.objecter ms_dispatch2 0x7f82a0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 3 client.4791.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 3 client.4791.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 20 client.4791.objecter dump_active .. 0 homeless 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f829c050070 con 0x7f82a015a5e0 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:02.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.322+0000 7f828dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:02.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.422+0000 7f82a79bc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:02.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.422+0000 7f82a79bc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:02.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.422+0000 7f82a79bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:02.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.422+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82a015a1e0 con 0x7f82a015a5e0 2026-02-24T18:37:02.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.454+0000 7f828dffb640 1 -- 10.20.193.18:0/279963592 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f829c0878a0 con 0x7f82a015a5e0 2026-02-24T18:37:02.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.454+0000 7f828dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:02.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.454+0000 7f828dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:02.457 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:02.457 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 10 librados: watch_flush enter 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 10 librados: watch_flush exit 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 20 client.4791.objecter shutdown clearing up homeless session... 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 10 client.4791.objecter successfully canceled tick 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 msgr2=0x7f827007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:02.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 0x7f827007b2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8290009c40 tx=0x7f8290009290 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 10 monclient: shutdown 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 msgr2=0x7f82a00c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c004b40 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 shutdown_connections 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8270078e30 0x7f827007b2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f82a015a5e0 0x7f82a00c2640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f82a00c99a0 0x7f82a00c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 --2- 10.20.193.18:0/279963592 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f82a006ff20 0x7f82a00c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 >> 10.20.193.18:0/279963592 conn(0x7f82a00cd780 msgr2=0x7f82a014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:02.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 shutdown_connections 2026-02-24T18:37:02.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 -- 10.20.193.18:0/279963592 wait complete. 2026-02-24T18:37:02.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:02.455+0000 7f82a79bc640 1 librados: shutdown 2026-02-24T18:37:05.469 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 10 monclient: get_monmap_and_config 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 10 monclient: build_initial_monmap 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 10 monclient: monmap: 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:05.539 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:05.537979+0000 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:05.537979+0000 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 1 Processor -- start 2026-02-24T18:37:05.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 1 -- start start 2026-02-24T18:37:05.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.537+0000 7fb5874da640 10 monclient: init 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient: _add_conns ranks=[3,0,1,2,5,4] 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb580155dd0 0x7fb580071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-a con 0x7fb580155dd0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb58015ffc0 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb5801561b0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb580073960 con 0x7fb58015ffc0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb580073c40 con 0x7fb5801561b0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb580073710 con 0x7fb580155dd0 2026-02-24T18:37:05.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb5874da640 10 monclient(hunting): authenticate will time out at 1569.219153s 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb580155dd0 0x7fb580071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb585a50640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fb5801561b0 0x7fb5800c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34372/0 (socket says 10.20.193.18:34372) 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb585a50640 1 -- 10.20.193.18:0/2276742526 learned_addr learned my addr 10.20.193.18:0/2276742526 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): get_auth_request con 0x7fb580155dd0 auth_method 0 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.538+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:05.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 10 monclient(hunting): handle_auth_done global_id 4794 payload 274 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 1 -- 10.20.193.18:0/2276742526 >> v1:10.20.193.32:6789/0 conn(0x7fb5801561b0 legacy=0x7fb5800c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 1 -- 10.20.193.18:0/2276742526 >> v1:10.20.193.18:6789/0 conn(0x7fb58015ffc0 legacy=0x7fb580076250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 10 monclient: _finish_hunting 0 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 1 monclient: found mon.noname-a 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.539+0000 7fb58524f640 1 -- 10.20.193.18:0/2276742526 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580073f20 con 0x7fb580155dd0 2026-02-24T18:37:05.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.540+0000 7fb58524f640 1 --2- 10.20.193.18:0/2276742526 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb580071360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098a0 tx=0x7fb57002ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a0cc48fd07c55c server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:05.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 1 -- 10.20.193.18:0/2276742526 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57003c070 con 0x7fb580155dd0 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: dump: 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:05.552 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: _finish_auth 0 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb5874da640 5 monclient: authenticate success, global_id 4794 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 1 -- 10.20.193.18:0/2276742526 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57002fa40 con 0x7fb580155dd0 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb5874da640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 1 -- 10.20.193.18:0/2276742526 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57002fe10 con 0x7fb580155dd0 2026-02-24T18:37:05.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb5874da640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb5874da640 4 set_mon_vals no callback set 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.542+0000 7fb56ffff640 10 monclient: dump: 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:05.554 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 10 monclient: shutdown 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 -- 10.20.193.18:0/2276742526 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 msgr2=0x7fb580071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 --2- 10.20.193.18:0/2276742526 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb580071360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098a0 tx=0x7fb57002ed80 comp rx=0 tx=0).stop 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 -- 10.20.193.18:0/2276742526 shutdown_connections 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 --2- 10.20.193.18:0/2276742526 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb580071360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 -- 10.20.193.18:0/2276742526 >> 10.20.193.18:0/2276742526 conn(0x7fb5800ca710 msgr2=0x7fb5800cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 -- 10.20.193.18:0/2276742526 shutdown_connections 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 1 -- 10.20.193.18:0/2276742526 wait complete. 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 10 monclient: build_initial_monmap 2026-02-24T18:37:05.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.543+0000 7fb5874da640 10 monclient: monmap: 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 librados: starting msgr at 2026-02-24T18:37:05.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 librados: starting objecter 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 Processor -- start 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 -- start start 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 librados: setting wanted keys 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 librados: calling monclient init 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient: init 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-a con 0x7fb580155dd0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb5801561b0 0x7fb58015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-b con 0x7fb5801561b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb58015ffc0 0x7fb5801f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:05.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): picked mon.noname-c con 0x7fb58015ffc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800c9730 con 0x7fb580155dd0 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800c8b00 con 0x7fb5801561b0 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800c98b0 con 0x7fb58015ffc0 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb5874da640 10 monclient(hunting): authenticate will time out at 1569.225609s 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55694/0 (socket says 10.20.193.18:55694) 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 1 -- 10.20.193.18:0/1458614773 learned_addr learned my addr 10.20.193.18:0/1458614773 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:05.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 10 monclient(hunting): get_auth_request con 0x7fb580155dd0 auth_method 0 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb58524f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.544+0000 7fb584a4e640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb5801561b0 0x7fb58015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb58015ffc0 0x7fb5801f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb58524f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): get_auth_request con 0x7fb5801561b0 auth_method 0 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:05.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): get_auth_request con 0x7fb58015ffc0 auth_method 0 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb584a4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.545+0000 7fb585a50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 10 monclient(hunting): handle_auth_done global_id 4797 payload 824 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 -- 10.20.193.18:0/1458614773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb58015ffc0 msgr2=0x7fb5801f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb58015ffc0 0x7fb5801f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 -- 10.20.193.18:0/1458614773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb5801561b0 msgr2=0x7fb58015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb5801561b0 0x7fb58015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 10 monclient: _finish_hunting 0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 monclient: found mon.noname-a 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.546+0000 7fb58524f640 1 -- 10.20.193.18:0/1458614773 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800c0580 con 0x7fb580155dd0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.547+0000 7fb58524f640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb570009130 tx=0x7fb5700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.547+0000 7fb56dffb640 10 client.?.objecter ms_handle_connect 0x7fb580155dd0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.547+0000 7fb56dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570046020 con 0x7fb580155dd0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:05.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: dump: 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:05.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5700426e0 con 0x7fb580155dd0 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57003c040 con 0x7fb580155dd0 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 5 monclient: authenticate success, global_id 4797 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _renew_subs 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56f7fe640 4 set_mon_vals no callback set 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5800c0860 con 0x7fb580155dd0 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:05.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb56dffb640 10 monclient: dump: 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 client.4797.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _renew_subs 2026-02-24T18:37:05.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5800c0e20 con 0x7fb580155dd0 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 1 librados: init done 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.549+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580071360 con 0x7fb580155dd0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.561+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb57003f030 con 0x7fb580155dd0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.561+0000 7fb56dffb640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 0x7fb55007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.562+0000 7fb584a4e640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 0x7fb55007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.562+0000 7fb584a4e640 10 monclient: get_auth_request con 0x7fb550078e00 auth_method 0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.562+0000 7fb56dffb640 10 client.4797.objecter ms_handle_connect 0x7fb550078e00 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.562+0000 7fb584a4e640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 0x7fb55007b2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cd9d0 tx=0x7fb5740091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.563+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fb570083380 con 0x7fb580155dd0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.563+0000 7fb56dffb640 10 client.4797.objecter ms_dispatch2 0x7fb580000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.563+0000 7fb56dffb640 3 client.4797.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.563+0000 7fb56dffb640 3 client.4797.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:05.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.563+0000 7fb56dffb640 20 client.4797.objecter dump_active .. 0 homeless 2026-02-24T18:37:05.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.662+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb570050030 con 0x7fb580155dd0 2026-02-24T18:37:05.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.662+0000 7fb56dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:05.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.662+0000 7fb56dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:05.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.761+0000 7fb5874da640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:05.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.761+0000 7fb5874da640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:05.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.761+0000 7fb5874da640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:05.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.761+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb5801597d0 con 0x7fb580155dd0 2026-02-24T18:37:05.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.763+0000 7fb56dffb640 1 -- 10.20.193.18:0/1458614773 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb570087340 con 0x7fb580155dd0 2026-02-24T18:37:05.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.763+0000 7fb56dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:05.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.763+0000 7fb56dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:05.765 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:05.766 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:05.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 10 librados: watch_flush enter 2026-02-24T18:37:05.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 10 librados: watch_flush exit 2026-02-24T18:37:05.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 20 client.4797.objecter shutdown clearing up homeless session... 2026-02-24T18:37:05.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 10 client.4797.objecter successfully canceled tick 2026-02-24T18:37:05.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 msgr2=0x7fb55007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 0x7fb55007b2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cd9d0 tx=0x7fb5740091c0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 10 monclient: shutdown 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 msgr2=0x7fb5800bff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.764+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb570009130 tx=0x7fb5700047c0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 shutdown_connections 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb550078e00 0x7fb55007b2c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb5801561b0 0x7fb58015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb58015ffc0 0x7fb5801f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 --2- 10.20.193.18:0/1458614773 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb580155dd0 0x7fb5800bff60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 >> 10.20.193.18:0/1458614773 conn(0x7fb5800ca710 msgr2=0x7fb580072c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:05.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 shutdown_connections 2026-02-24T18:37:05.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 -- 10.20.193.18:0/1458614773 wait complete. 2026-02-24T18:37:05.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:05.765+0000 7fb5874da640 1 librados: shutdown 2026-02-24T18:37:08.776 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:08.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 10 monclient: get_monmap_and_config 2026-02-24T18:37:08.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 10 monclient: build_initial_monmap 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 10 monclient: monmap: 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:08.847346+0000 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:08.847346+0000 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:08.849 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 1 Processor -- start 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 1 -- start start 2026-02-24T18:37:08.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.846+0000 7f161ce74640 10 monclient: init 2026-02-24T18:37:08.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient: _add_conns ranks=[4,5,1,3,0,2] 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f16181527d0 0x7f161806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-b con 0x7f16181527d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1618152bb0 0x7f16180c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-c con 0x7f1618152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f16180c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161806e300 con 0x7f16180c5fc0 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f161806f1f0 con 0x7f16181527d0 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:08.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f16180c6470 con 0x7f1618152bb0 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f161ce74640 10 monclient(hunting): authenticate will time out at 1572.528464s 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f1615d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1618152bb0 0x7f16180c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.847+0000 7f1616575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f16181527d0 0x7f161806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1618152bb0 0x7f16180c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:46274/0 (socket says 10.20.193.18:46274) 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 1 -- 10.20.193.18:0/3774160460 learned_addr learned my addr 10.20.193.18:0/3774160460 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): get_auth_request con 0x7f1618152bb0 auth_method 0 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1616575640 10 monclient(hunting): get_auth_request con 0x7f16181527d0 auth_method 0 2026-02-24T18:37:08.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1616575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1616575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1616575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.848+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 10 monclient(hunting): handle_auth_done global_id 4760 payload 274 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 1 -- 10.20.193.18:0/3774160460 >> v2:10.20.193.32:3300/0 conn(0x7f16181527d0 msgr2=0x7f161806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 1 --2- 10.20.193.18:0/3774160460 >> v2:10.20.193.32:3300/0 conn(0x7f16181527d0 0x7f161806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 1 -- 10.20.193.18:0/3774160460 >> v1:10.20.193.32:6789/0 conn(0x7f16180c5fc0 legacy=0x7f16180c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:08.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 10 monclient: _finish_hunting 0 2026-02-24T18:37:08.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 1 monclient: found mon.noname-c 2026-02-24T18:37:08.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:08.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:08.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.849+0000 7f1615d74640 1 -- 10.20.193.18:0/3774160460 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161806e5e0 con 0x7f1618152bb0 2026-02-24T18:37:08.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.850+0000 7f1615d74640 1 --2- 10.20.193.18:0/3774160460 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1618152bb0 0x7f16180c5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f16180750e0 tx=0x7f160002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bd74c66f8914b9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:08.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 1 -- 10.20.193.18:0/3774160460 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160003c070 con 0x7f1618152bb0 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: dump: 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:08.863 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: _finish_auth 0 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: _check_auth_tickets 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 1 -- 10.20.193.18:0/3774160460 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160002fbe0 con 0x7f1618152bb0 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 1 -- 10.20.193.18:0/3774160460 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600037440 con 0x7f1618152bb0 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f1615573640 10 monclient: dump: 2026-02-24T18:37:08.864 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 5 monclient: authenticate success, global_id 4760 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 4 set_mon_vals no callback set 2026-02-24T18:37:08.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 10 monclient: shutdown 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 1 -- 10.20.193.18:0/3774160460 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1618152bb0 msgr2=0x7f16180c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.852+0000 7f161ce74640 1 --2- 10.20.193.18:0/3774160460 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1618152bb0 0x7f16180c5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f16180750e0 tx=0x7f160002eed0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 -- 10.20.193.18:0/3774160460 shutdown_connections 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 --2- 10.20.193.18:0/3774160460 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1618152bb0 0x7f16180c5a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 --2- 10.20.193.18:0/3774160460 >> v2:10.20.193.32:3300/0 conn(0x7f16181527d0 0x7f161806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 -- 10.20.193.18:0/3774160460 >> 10.20.193.18:0/3774160460 conn(0x7f16180cd780 msgr2=0x7f16180cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 -- 10.20.193.18:0/3774160460 shutdown_connections 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 -- 10.20.193.18:0/3774160460 wait complete. 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 10 monclient: build_initial_monmap 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 10 monclient: monmap: 2026-02-24T18:37:08.866 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 librados: starting msgr at 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 librados: starting objecter 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 Processor -- start 2026-02-24T18:37:08.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 -- start start 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 librados: setting wanted keys 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 1 librados: calling monclient init 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.853+0000 7f161ce74640 10 monclient: init 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 0x7f16181f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-c con 0x7f16180c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16181527d0 0x7f16181476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-a con 0x7f16181527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 0x7f16181531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): picked mon.noname-b con 0x7f1618152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:08.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f161806ee80 con 0x7f16181527d0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f161806eba0 con 0x7f1618152bb0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f161806dfa0 con 0x7f16180c5fc0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f161ce74640 10 monclient(hunting): authenticate will time out at 1572.535282s 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 0x7f16181f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 0x7f16181531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1615d74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16181527d0 0x7f16181476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 0x7f16181f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:46280/0 (socket says 10.20.193.18:46280) 2026-02-24T18:37:08.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 1 -- 10.20.193.18:0/1788710501 learned_addr learned my addr 10.20.193.18:0/1788710501 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): get_auth_request con 0x7f16180c5fc0 auth_method 0 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): get_auth_request con 0x7f1618152bb0 auth_method 0 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1615d74640 10 monclient(hunting): get_auth_request con 0x7f16181527d0 auth_method 0 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1615d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1615d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1615d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:08.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.854+0000 7f1616d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.855+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.855+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.855+0000 7f1615d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 10 monclient(hunting): handle_auth_done global_id 4780 payload 824 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 -- 10.20.193.18:0/1788710501 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 msgr2=0x7f16181f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 0x7f16181f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 -- 10.20.193.18:0/1788710501 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16181527d0 msgr2=0x7f16181476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:08.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16181527d0 0x7f16181476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 10 monclient: _finish_hunting 0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 monclient: found mon.noname-b 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.856+0000 7f1616d76640 1 -- 10.20.193.18:0/1788710501 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618155020 con 0x7f1618152bb0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.857+0000 7f1616d76640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 0x7f16181531c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f160800cbf0 tx=0x7f160800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.857+0000 7f15feffd640 10 client.?.objecter ms_handle_connect 0x7f1618152bb0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.857+0000 7f15feffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160800ce10 con 0x7f1618152bb0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:08.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: dump: 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: _finish_auth 0 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: _check_auth_tickets 2026-02-24T18:37:08.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1608003a30 con 0x7f1618152bb0 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 5 monclient: authenticate success, global_id 4780 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _renew_subs 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16181552c0 con 0x7f1618152bb0 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608003e00 con 0x7f1618152bb0 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f15feffd640 10 monclient: dump: 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:08.874 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f1614d72640 4 set_mon_vals no callback set 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 client.4780.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _renew_subs 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1618155880 con 0x7f1618152bb0 2026-02-24T18:37:08.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 1 librados: init done 2026-02-24T18:37:08.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:08.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:08.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:08.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.859+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161806da60 con 0x7f1618152bb0 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.873+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1608005090 con 0x7f1618152bb0 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.874+0000 7f15feffd640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 0x7f15e407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.874+0000 7f1616575640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 0x7f15e407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.874+0000 7f1616575640 10 monclient: get_auth_request con 0x7f15e4078e00 auth_method 0 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.874+0000 7f15feffd640 10 client.4780.objecter ms_handle_connect 0x7f15e4078e00 2026-02-24T18:37:08.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.874+0000 7f1616575640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 0x7f15e407b2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d00 tx=0x7f160c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:08.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.914+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f160805e740 con 0x7f1618152bb0 2026-02-24T18:37:08.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.914+0000 7f15feffd640 10 client.4780.objecter ms_dispatch2 0x7f1618000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:08.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.914+0000 7f15feffd640 3 client.4780.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:08.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.914+0000 7f15feffd640 3 client.4780.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:08.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.914+0000 7f15feffd640 20 client.4780.objecter dump_active .. 0 homeless 2026-02-24T18:37:08.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.971+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1608062700 con 0x7f1618152bb0 2026-02-24T18:37:08.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.971+0000 7f15feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:08.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:08.971+0000 7f15feffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:09.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.073+0000 7f161ce74640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:09.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.074+0000 7f161ce74640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:09.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.074+0000 7f161ce74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:09.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.074+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16180c8120 con 0x7f1618152bb0 2026-02-24T18:37:09.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.076+0000 7f15feffd640 1 -- 10.20.193.18:0/1788710501 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1608066a00 con 0x7f1618152bb0 2026-02-24T18:37:09.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.076+0000 7f15feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:09.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.076+0000 7f15feffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:09.078 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:09.078 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:09.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 10 librados: watch_flush enter 2026-02-24T18:37:09.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 10 librados: watch_flush exit 2026-02-24T18:37:09.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 20 client.4780.objecter shutdown clearing up homeless session... 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 10 client.4780.objecter successfully canceled tick 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 msgr2=0x7f15e407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 0x7f15e407b2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d00 tx=0x7f160c009290 comp rx=0 tx=0).stop 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 10 monclient: shutdown 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 msgr2=0x7f16181531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 0x7f16181531c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f160800cbf0 tx=0x7f160800a430 comp rx=0 tx=0).stop 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 shutdown_connections 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f15e4078e00 0x7f15e407b2c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1618152bb0 0x7f16181531c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:09.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16181527d0 0x7f16181476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:09.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 --2- 10.20.193.18:0/1788710501 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16180c5fc0 0x7f16181f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:09.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 >> 10.20.193.18:0/1788710501 conn(0x7f16180cd780 msgr2=0x7f16180cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:09.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 shutdown_connections 2026-02-24T18:37:09.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 -- 10.20.193.18:0/1788710501 wait complete. 2026-02-24T18:37:09.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:09.078+0000 7f161ce74640 1 librados: shutdown 2026-02-24T18:37:12.090 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: get_monmap_and_config 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: build_initial_monmap 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: monmap: 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:12.160875+0000 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:12.160875+0000 2026-02-24T18:37:12.162 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 Processor -- start 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 -- start start 2026-02-24T18:37:12.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: init 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient: _add_conns ranks=[1,3,4,2,5,0] 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc8ec1527d0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-a con 0x7fc8ec152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc8ec0c5fc0 0x7fc8ec0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-b con 0x7fc8ec0c5fc0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ec06e300 con 0x7fc8ec1527d0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc8ec06f1f0 con 0x7fc8ec152bb0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc8ec0c6470 con 0x7fc8ec0c5fc0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.160+0000 7fc8f44a0640 10 monclient(hunting): authenticate will time out at 1575.841998s 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59172/0 (socket says 10.20.193.18:59172) 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 1 -- 10.20.193.18:0/21347794 learned_addr learned my addr 10.20.193.18:0/21347794 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): get_auth_request con 0x7fc8ec152bb0 auth_method 0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.161+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f2a16640 1 --2- 10.20.193.18:0/21347794 >> v2:10.20.193.32:3300/0 conn(0x7fc8ec0c5fc0 0x7fc8ec0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f2a16640 10 monclient(hunting): get_auth_request con 0x7fc8ec0c5fc0 auth_method 0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f2a16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f2a16640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f2a16640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_done global_id 4803 payload 274 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 1 -- 10.20.193.18:0/21347794 >> v2:10.20.193.32:3300/0 conn(0x7fc8ec0c5fc0 msgr2=0x7fc8ec0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 1 --2- 10.20.193.18:0/21347794 >> v2:10.20.193.32:3300/0 conn(0x7fc8ec0c5fc0 0x7fc8ec0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 1 -- 10.20.193.18:0/21347794 >> v1:10.20.193.32:6789/0 conn(0x7fc8ec1527d0 legacy=0x7fc8ec06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 10 monclient: _finish_hunting 0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 1 monclient: found mon.noname-a 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.162+0000 7fc8f1a14640 1 -- 10.20.193.18:0/21347794 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec06e5e0 con 0x7fc8ec152bb0 2026-02-24T18:37:12.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.163+0000 7fc8f1a14640 1 --2- 10.20.193.18:0/21347794 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec158940 tx=0x7fc8e002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b51d5c88612c35 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 1 -- 10.20.193.18:0/21347794 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e003c070 con 0x7fc8ec152bb0 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: dump: 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: _finish_auth 0 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: _check_auth_tickets 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 1 -- 10.20.193.18:0/21347794 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e002fbe0 con 0x7fc8ec152bb0 2026-02-24T18:37:12.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 1 -- 10.20.193.18:0/21347794 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0037440 con 0x7fc8ec152bb0 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f44a0640 5 monclient: authenticate success, global_id 4803 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f44a0640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f44a0640 4 set_mon_vals no callback set 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.165+0000 7fc8f1213640 10 monclient: dump: 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 10 monclient: shutdown 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 -- 10.20.193.18:0/21347794 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 msgr2=0x7fc8ec0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/21347794 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec158940 tx=0x7fc8e002eed0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 -- 10.20.193.18:0/21347794 shutdown_connections 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/21347794 >> v2:10.20.193.32:3300/0 conn(0x7fc8ec0c5fc0 0x7fc8ec0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/21347794 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec0c5a80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 -- 10.20.193.18:0/21347794 >> 10.20.193.18:0/21347794 conn(0x7fc8ec0cd780 msgr2=0x7fc8ec0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 -- 10.20.193.18:0/21347794 shutdown_connections 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 -- 10.20.193.18:0/21347794 wait complete. 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 10 monclient: build_initial_monmap 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 10 monclient: monmap: 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:12.169 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 librados: starting msgr at 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 librados: starting objecter 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.166+0000 7fc8f44a0640 1 Processor -- start 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 -- start start 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 librados: setting wanted keys 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 librados: calling monclient init 2026-02-24T18:37:12.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient: init 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc8ec0c5fc0 0x7fc8ec14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-b con 0x7fc8ec0c5fc0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8ec1527d0 0x7fc8ec148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-c con 0x7fc8ec1527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): picked mon.noname-a con 0x7fc8ec152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec06ee80 con 0x7fc8ec152bb0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec06eba0 con 0x7fc8ec0c5fc0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0c93a0 con 0x7fc8ec1527d0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f44a0640 10 monclient(hunting): authenticate will time out at 1575.848433s 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2215640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc8ec0c5fc0 0x7fc8ec14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59178/0 (socket says 10.20.193.18:59178) 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 1 -- 10.20.193.18:0/4263821462 learned_addr learned my addr 10.20.193.18:0/4263821462 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 10 monclient(hunting): get_auth_request con 0x7fc8ec152bb0 auth_method 0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2a16640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f1a14640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8ec1527d0 0x7fc8ec148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2215640 10 monclient(hunting): get_auth_request con 0x7fc8ec0c5fc0 auth_method 0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2215640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.167+0000 7fc8f2215640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): get_auth_request con 0x7fc8ec1527d0 auth_method 0 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2a16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2a16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2a16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:12.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2215640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2215640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f2215640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.168+0000 7fc8f1a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 10 monclient(hunting): handle_auth_done global_id 4806 payload 824 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 -- 10.20.193.18:0/4263821462 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8ec1527d0 msgr2=0x7fc8ec148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8ec1527d0 0x7fc8ec148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 -- 10.20.193.18:0/4263821462 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc8ec0c5fc0 msgr2=0x7fc8ec14ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc8ec0c5fc0 0x7fc8ec14ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 10 monclient: _finish_hunting 0 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 monclient: found mon.noname-a 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.169+0000 7fc8f2a16640 1 -- 10.20.193.18:0/4263821462 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec14dca0 con 0x7fc8ec152bb0 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.170+0000 7fc8f2a16640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007e40 tx=0x7fc8e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.170+0000 7fc8d2ffd640 10 client.?.objecter ms_handle_connect 0x7fc8ec152bb0 2026-02-24T18:37:12.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.170+0000 7fc8d2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:12.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8014620 con 0x7fc8ec152bb0 2026-02-24T18:37:12.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: dump: 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e8019070 con 0x7fc8ec152bb0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8014a20 con 0x7fc8ec152bb0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 5 monclient: authenticate success, global_id 4806 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f0a12640 4 set_mon_vals no callback set 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8d2ffd640 10 monclient: dump: 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _renew_subs 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec14df80 con 0x7fc8ec152bb0 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 client.4806.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:12.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _renew_subs 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec14e540 con 0x7fc8ec152bb0 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 1 librados: init done 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.172+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec14e920 con 0x7fc8ec152bb0 2026-02-24T18:37:12.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.183+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc8e8014bc0 con 0x7fc8ec152bb0 2026-02-24T18:37:12.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.184+0000 7fc8d2ffd640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 0x7fc8bc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:12.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.184+0000 7fc8f2215640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 0x7fc8bc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:12.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.184+0000 7fc8f2215640 10 monclient: get_auth_request con 0x7fc8bc078e50 auth_method 0 2026-02-24T18:37:12.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.185+0000 7fc8f2215640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 0x7fc8bc07b310 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec1f8130 tx=0x7fc8dc007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:12.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.185+0000 7fc8d2ffd640 10 client.4806.objecter ms_handle_connect 0x7fc8bc078e50 2026-02-24T18:37:12.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.186+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fc8e805f040 con 0x7fc8ec152bb0 2026-02-24T18:37:12.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.186+0000 7fc8d2ffd640 10 client.4806.objecter ms_dispatch2 0x7fc8ec000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:12.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.186+0000 7fc8d2ffd640 3 client.4806.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:12.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.186+0000 7fc8d2ffd640 3 client.4806.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:12.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.186+0000 7fc8d2ffd640 20 client.4806.objecter dump_active .. 0 homeless 2026-02-24T18:37:12.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.288+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc8ec14e920 con 0x7fc8ec152bb0 2026-02-24T18:37:12.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.288+0000 7fc8d2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:12.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.288+0000 7fc8d2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:12.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.389+0000 7fc8f44a0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:12.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.389+0000 7fc8f44a0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:12.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.389+0000 7fc8f44a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:12.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.389+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc8ec0c9730 con 0x7fc8ec152bb0 2026-02-24T18:37:12.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.392+0000 7fc8d2ffd640 1 -- 10.20.193.18:0/4263821462 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc8e8063000 con 0x7fc8ec152bb0 2026-02-24T18:37:12.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.392+0000 7fc8d2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:12.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.392+0000 7fc8d2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:12.393 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:12.393 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:12.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 10 librados: watch_flush enter 2026-02-24T18:37:12.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 10 librados: watch_flush exit 2026-02-24T18:37:12.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 20 client.4806.objecter shutdown clearing up homeless session... 2026-02-24T18:37:12.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 10 client.4806.objecter successfully canceled tick 2026-02-24T18:37:12.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 msgr2=0x7fc8bc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 0x7fc8bc07b310 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec1f8130 tx=0x7fc8dc007f30 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 10 monclient: shutdown 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 msgr2=0x7fc8ec1f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007e40 tx=0x7fc8e800e4b0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 shutdown_connections 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc8bc078e50 0x7fc8bc07b310 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc8ec152bb0 0x7fc8ec1f71e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc8ec1527d0 0x7fc8ec148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 --2- 10.20.193.18:0/4263821462 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc8ec0c5fc0 0x7fc8ec14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 >> 10.20.193.18:0/4263821462 conn(0x7fc8ec0cd780 msgr2=0x7fc8ec06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 shutdown_connections 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 -- 10.20.193.18:0/4263821462 wait complete. 2026-02-24T18:37:12.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:12.393+0000 7fc8f44a0640 1 librados: shutdown 2026-02-24T18:37:15.403 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:15.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.474+0000 7fd4d3857640 10 monclient: get_monmap_and_config 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.474+0000 7fd4d3857640 10 monclient: build_initial_monmap 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.474+0000 7fd4d3857640 10 monclient: monmap: 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:15.475810+0000 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:15.475810+0000 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:15.479 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 Processor -- start 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 -- start start 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient: init 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient: _add_conns ranks=[2,3,4,0,5,1] 2026-02-24T18:37:15.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd4cc06ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd4cc0c99a0 0x7fd4cc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-a con 0x7fd4cc0c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fd4cc15a5e0 0x7fd4cc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-b con 0x7fd4cc15a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc0c7e90 con 0x7fd4cc06ff20 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc159a40 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc0c8170 con 0x7fd4cc15a5e0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.475+0000 7fd4d3857640 10 monclient(hunting): authenticate will time out at 1579.156915s 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd4cc0c99a0 0x7fd4cc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd4cc0c99a0 0x7fd4cc070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59182/0 (socket says 10.20.193.18:59182) 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/28969029 learned_addr learned my addr 10.20.193.18:0/28969029 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): get_auth_request con 0x7fd4cc0c99a0 auth_method 0 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d1dcd640 1 --2- 10.20.193.18:0/28969029 >> v2:10.20.193.32:3300/0 conn(0x7fd4cc15a5e0 0x7fd4cc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d1dcd640 10 monclient(hunting): get_auth_request con 0x7fd4cc15a5e0 auth_method 0 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d1dcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:15.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d1dcd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:15.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.476+0000 7fd4d1dcd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:15.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:15.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:15.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:15.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_done global_id 4809 payload 274 2026-02-24T18:37:15.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/28969029 >> v2:10.20.193.32:3300/0 conn(0x7fd4cc15a5e0 msgr2=0x7fd4cc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:15.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 1 --2- 10.20.193.18:0/28969029 >> v2:10.20.193.32:3300/0 conn(0x7fd4cc15a5e0 0x7fd4cc14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/28969029 >> v1:10.20.193.110:6789/0 conn(0x7fd4cc06ff20 legacy=0x7fd4cc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 10 monclient: _finish_hunting 0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 1 monclient: found mon.noname-a 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.477+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/28969029 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0c8450 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.478+0000 7fd4d0dcb640 1 --2- 10.20.193.18:0/28969029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc070300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009970 tx=0x7fd4c002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aac0d67c0997bab server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 1 -- 10.20.193.18:0/28969029 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c003c070 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 10 monclient: dump: 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:15.491 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 10 monclient: _finish_auth 0 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4d3857640 5 monclient: authenticate success, global_id 4809 2026-02-24T18:37:15.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.479+0000 7fd4d3857640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 1 -- 10.20.193.18:0/28969029 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c002fbc0 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 1 -- 10.20.193.18:0/28969029 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037440 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4bbfff640 10 monclient: dump: 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.493 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 4 set_mon_vals no callback set 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 10 monclient: shutdown 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 -- 10.20.193.18:0/28969029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 msgr2=0x7fd4cc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 --2- 10.20.193.18:0/28969029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc070300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009970 tx=0x7fd4c002eef0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 -- 10.20.193.18:0/28969029 shutdown_connections 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 --2- 10.20.193.18:0/28969029 >> v2:10.20.193.32:3300/0 conn(0x7fd4cc15a5e0 0x7fd4cc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 --2- 10.20.193.18:0/28969029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc070300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 -- 10.20.193.18:0/28969029 >> 10.20.193.18:0/28969029 conn(0x7fd4cc0cd780 msgr2=0x7fd4cc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 -- 10.20.193.18:0/28969029 shutdown_connections 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.480+0000 7fd4d3857640 1 -- 10.20.193.18:0/28969029 wait complete. 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 10 monclient: build_initial_monmap 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 10 monclient: monmap: 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:15.495 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 librados: starting msgr at 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 librados: starting objecter 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 Processor -- start 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 -- start start 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 librados: setting wanted keys 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 1 librados: calling monclient init 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.481+0000 7fd4d3857640 10 monclient: init 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd4cc06ff20 0x7fd4cc0c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-c con 0x7fd4cc06ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-a con 0x7fd4cc0c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd4cc15a5e0 0x7fd4cc14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): picked mon.noname-b con 0x7fd4cc15a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc150350 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0c8a10 con 0x7fd4cc15a5e0 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1504d0 con 0x7fd4cc06ff20 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:15.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d3857640 10 monclient(hunting): authenticate will time out at 1579.163080s 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59190/0 (socket says 10.20.193.18:59190) 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/4155743881 learned_addr learned my addr 10.20.193.18:0/4155743881 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): get_auth_request con 0x7fd4cc0c99a0 auth_method 0 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d15cc640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd4cc06ff20 0x7fd4cc0c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d1dcd640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd4cc15a5e0 0x7fd4cc14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:15.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d15cc640 10 monclient(hunting): get_auth_request con 0x7fd4cc06ff20 auth_method 0 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d15cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d15cc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d15cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d1dcd640 10 monclient(hunting): get_auth_request con 0x7fd4cc15a5e0 auth_method 0 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d1dcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d1dcd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.482+0000 7fd4d1dcd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d15cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d15cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d15cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:15.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.483+0000 7fd4d1dcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 10 monclient(hunting): handle_auth_done global_id 4812 payload 824 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/4155743881 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd4cc06ff20 msgr2=0x7fd4cc0c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd4cc06ff20 0x7fd4cc0c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/4155743881 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd4cc15a5e0 msgr2=0x7fd4cc14d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd4cc15a5e0 0x7fd4cc14d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 10 monclient: _finish_hunting 0 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 monclient: found mon.noname-a 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.484+0000 7fd4d0dcb640 1 -- 10.20.193.18:0/4155743881 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0c2ba0 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.485+0000 7fd4d0dcb640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002f4a0 tx=0x7fd4c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.485+0000 7fd4b9ffb640 10 client.?.objecter ms_handle_connect 0x7fd4cc0c99a0 2026-02-24T18:37:15.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.485+0000 7fd4b9ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c003c040 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: dump: 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:15.501 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0042410 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 5 monclient: authenticate success, global_id 4812 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 10 monclient: _renew_subs 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc0c2e80 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00427e0 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4bb7fe640 4 set_mon_vals no callback set 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4b9ffb640 10 monclient: dump: 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:15.503 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 10 client.4812.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 10 monclient: _renew_subs 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc0c3440 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.486+0000 7fd4d3857640 1 librados: init done 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.487+0000 7fd4d3857640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.487+0000 7fd4d3857640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.487+0000 7fd4d3857640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.487+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc150860 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.594+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd4c0042980 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.595+0000 7fd4b9ffb640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 0x7fd49c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:15.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.595+0000 7fd4d15cc640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 0x7fd49c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:15.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.595+0000 7fd4d15cc640 10 monclient: get_auth_request con 0x7fd49c078e50 auth_method 0 2026-02-24T18:37:15.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.595+0000 7fd4b9ffb640 10 client.4812.objecter ms_handle_connect 0x7fd49c078e50 2026-02-24T18:37:15.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.595+0000 7fd4d15cc640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 0x7fd49c07b310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0051f0 tx=0x7fd4bc007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd4c0083650 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 10 client.4812.objecter ms_dispatch2 0x7fd4cc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 3 client.4812.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 3 client.4812.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 20 client.4812.objecter dump_active .. 0 homeless 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd4c0087610 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:15.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.603+0000 7fd4b9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:15.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.702+0000 7fd4d3857640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:15.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.702+0000 7fd4d3857640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:15.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.702+0000 7fd4d3857640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:15.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.702+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd4cc1f35c0 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.705+0000 7fd4b9ffb640 1 -- 10.20.193.18:0/4155743881 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd4c008b910 con 0x7fd4cc0c99a0 2026-02-24T18:37:15.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.705+0000 7fd4b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:15.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.705+0000 7fd4b9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:15.707 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:15.707 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 10 librados: watch_flush enter 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 10 librados: watch_flush exit 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 20 client.4812.objecter shutdown clearing up homeless session... 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 10 client.4812.objecter successfully canceled tick 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 msgr2=0x7fd49c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 0x7fd49c07b310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0051f0 tx=0x7fd4bc007f30 comp rx=0 tx=0).stop 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 10 monclient: shutdown 2026-02-24T18:37:15.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 msgr2=0x7fd4cc1f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002f4a0 tx=0x7fd4c00047c0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 shutdown_connections 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd49c078e50 0x7fd49c07b310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd4cc15a5e0 0x7fd4cc14d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd4cc0c99a0 0x7fd4cc1f3080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 --2- 10.20.193.18:0/4155743881 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd4cc06ff20 0x7fd4cc0c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 >> 10.20.193.18:0/4155743881 conn(0x7fd4cc0cd780 msgr2=0x7fd4cc0c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 shutdown_connections 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 -- 10.20.193.18:0/4155743881 wait complete. 2026-02-24T18:37:15.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:15.706+0000 7fd4d3857640 1 librados: shutdown 2026-02-24T18:37:18.718 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.785+0000 7fa98a261640 10 monclient: get_monmap_and_config 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.785+0000 7fa98a261640 10 monclient: build_initial_monmap 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.785+0000 7fa98a261640 10 monclient: monmap: 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:18.786697+0000 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:18.786697+0000 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:18.787 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 Processor -- start 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 -- start start 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient: init 2026-02-24T18:37:18.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient: _add_conns ranks=[5,4,0,3,2,1] 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa98406ff20 0x7fa984070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-c con 0x7fa98406ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa9840c99a0 0x7fa984159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-b con 0x7fa9840c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa98415a410 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98415a860 con 0x7fa98415a410 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa984159a40 con 0x7fa9840c99a0 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa9840c7e90 con 0x7fa98406ff20 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.786+0000 7fa98a261640 10 monclient(hunting): authenticate will time out at 1582.467989s 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa983fff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa98415a410 0x7fa98414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:47830/0 (socket says 10.20.193.18:47830) 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa983fff640 1 -- 10.20.193.18:0/423029005 learned_addr learned my addr 10.20.193.18:0/423029005 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa982ffd640 1 --2- 10.20.193.18:0/423029005 >> v2:10.20.193.32:3300/0 conn(0x7fa9840c99a0 0x7fa984159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa9837fe640 1 --2- 10.20.193.18:0/423029005 >> v2:10.20.193.110:3300/0 conn(0x7fa98406ff20 0x7fa984070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa9837fe640 10 monclient(hunting): get_auth_request con 0x7fa98406ff20 auth_method 0 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa9837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa9837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa9837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa982ffd640 10 monclient(hunting): get_auth_request con 0x7fa9840c99a0 auth_method 0 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa982ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa982ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.787+0000 7fa982ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 1 -- 10.20.193.18:0/423029005 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946116582 0 0) 0x7fa98415a860 con 0x7fa98415a410 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 10 monclient(hunting): _negotiate 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 10 monclient(hunting): my global_id is 4815 2026-02-24T18:37:18.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.788+0000 7fa9827fc640 1 -- 10.20.193.18:0/423029005 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa968003620 con 0x7fa98415a410 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 10 monclient(hunting): handle_auth_done global_id 4798 payload 274 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 1 -- 10.20.193.18:0/423029005 >> v2:10.20.193.110:3300/0 conn(0x7fa98406ff20 msgr2=0x7fa984070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 1 --2- 10.20.193.18:0/423029005 >> v2:10.20.193.110:3300/0 conn(0x7fa98406ff20 0x7fa984070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 1 -- 10.20.193.18:0/423029005 >> v1:10.20.193.18:6789/0 conn(0x7fa98415a410 legacy=0x7fa98414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 1 monclient: found mon.noname-b 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.789+0000 7fa982ffd640 1 -- 10.20.193.18:0/423029005 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9840c8170 con 0x7fa9840c99a0 2026-02-24T18:37:18.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.790+0000 7fa982ffd640 1 --2- 10.20.193.18:0/423029005 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa984159ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740098a0 tx=0x7fa97402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca2889ab2e495e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 1 -- 10.20.193.18:0/423029005 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97403c070 con 0x7fa9840c99a0 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: dump: 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.792 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 5 monclient: authenticate success, global_id 4798 2026-02-24T18:37:18.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 1 -- 10.20.193.18:0/423029005 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97402fae0 con 0x7fa9840c99a0 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 1 -- 10.20.193.18:0/423029005 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037440 con 0x7fa9840c99a0 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa9827fc640 10 monclient: dump: 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 4 set_mon_vals no callback set 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 10 monclient: shutdown 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 1 -- 10.20.193.18:0/423029005 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 msgr2=0x7fa984159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.791+0000 7fa98a261640 1 --2- 10.20.193.18:0/423029005 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa984159ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740098a0 tx=0x7fa97402ee10 comp rx=0 tx=0).stop 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 -- 10.20.193.18:0/423029005 shutdown_connections 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 --2- 10.20.193.18:0/423029005 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa984159ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 --2- 10.20.193.18:0/423029005 >> v2:10.20.193.110:3300/0 conn(0x7fa98406ff20 0x7fa984070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 -- 10.20.193.18:0/423029005 >> 10.20.193.18:0/423029005 conn(0x7fa9840cd780 msgr2=0x7fa9840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 -- 10.20.193.18:0/423029005 shutdown_connections 2026-02-24T18:37:18.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 -- 10.20.193.18:0/423029005 wait complete. 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 10 monclient: build_initial_monmap 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 10 monclient: monmap: 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 librados: starting msgr at 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 librados: starting objecter 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 Processor -- start 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 -- start start 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 librados: setting wanted keys 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 1 librados: calling monclient init 2026-02-24T18:37:18.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.792+0000 7fa98a261640 10 monclient: init 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-a con 0x7fa98406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa98414cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-b con 0x7fa9840c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa98415a410 0x7fa98414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): picked mon.noname-c con 0x7fa98415a410 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840c8a10 con 0x7fa98406ff20 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840c8730 con 0x7fa9840c99a0 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:18.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa984150520 con 0x7fa98415a410 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa98a261640 10 monclient(hunting): authenticate will time out at 1582.474258s 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa982ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa98414cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59198/0 (socket says 10.20.193.18:59198) 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 1 -- 10.20.193.18:0/1761841590 learned_addr learned my addr 10.20.193.18:0/1761841590 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): get_auth_request con 0x7fa98406ff20 auth_method 0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa982ffd640 10 monclient(hunting): get_auth_request con 0x7fa9840c99a0 auth_method 0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa983fff640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa98415a410 0x7fa98414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa982ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa982ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa982ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa9837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa983fff640 10 monclient(hunting): get_auth_request con 0x7fa98415a410 auth_method 0 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa983fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa983fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.793+0000 7fa983fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa982ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa983fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa983fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:18.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.794+0000 7fa983fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 10 monclient(hunting): handle_auth_done global_id 4818 payload 824 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 -- 10.20.193.18:0/1761841590 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa98415a410 msgr2=0x7fa98414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa98415a410 0x7fa98414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 -- 10.20.193.18:0/1761841590 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 msgr2=0x7fa98414cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa98414cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 monclient: found mon.noname-a 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.795+0000 7fa9837fe640 1 -- 10.20.193.18:0/1761841590 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9841f3910 con 0x7fa98406ff20 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.796+0000 7fa9837fe640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa97000ebe0 tx=0x7fa97000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.796+0000 7fa95ffff640 10 client.?.objecter ms_handle_connect 0x7fa98406ff20 2026-02-24T18:37:18.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.796+0000 7fa95ffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970017070 con 0x7fa98406ff20 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: dump: 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: _finish_auth 0 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97000cc40 con 0x7fa98406ff20 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 5 monclient: authenticate success, global_id 4818 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _renew_subs 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9841f3bf0 con 0x7fa98406ff20 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970013470 con 0x7fa98406ff20 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa981ffb640 4 set_mon_vals no callback set 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa95ffff640 10 monclient: dump: 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:18.800 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 client.4818.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _renew_subs 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9841f41b0 con 0x7fa98406ff20 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 1 librados: init done 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:18.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.798+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa984070300 con 0x7fa98406ff20 2026-02-24T18:37:18.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.809+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa970013610 con 0x7fa98406ff20 2026-02-24T18:37:18.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.810+0000 7fa95ffff640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 0x7fa95407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:18.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.810+0000 7fa982ffd640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 0x7fa95407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:18.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.810+0000 7fa982ffd640 10 monclient: get_auth_request con 0x7fa954078e00 auth_method 0 2026-02-24T18:37:18.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.811+0000 7fa95ffff640 10 client.4818.objecter ms_handle_connect 0x7fa954078e00 2026-02-24T18:37:18.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.811+0000 7fa982ffd640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 0x7fa95407b2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa974039000 tx=0x7fa974033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:18.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.812+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa97005e570 con 0x7fa98406ff20 2026-02-24T18:37:18.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.812+0000 7fa95ffff640 10 client.4818.objecter ms_dispatch2 0x7fa984000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:18.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.812+0000 7fa95ffff640 3 client.4818.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:18.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.812+0000 7fa95ffff640 3 client.4818.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:18.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.812+0000 7fa95ffff640 20 client.4818.objecter dump_active .. 0 homeless 2026-02-24T18:37:18.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.913+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa970062530 con 0x7fa98406ff20 2026-02-24T18:37:18.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.913+0000 7fa95ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:18.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:18.913+0000 7fa95ffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:19.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.012+0000 7fa98a261640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:19.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.012+0000 7fa98a261640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:19.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.012+0000 7fa98a261640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:19.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.012+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa9840cdd00 con 0x7fa98406ff20 2026-02-24T18:37:19.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.014+0000 7fa95ffff640 1 -- 10.20.193.18:0/1761841590 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa970066830 con 0x7fa98406ff20 2026-02-24T18:37:19.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.014+0000 7fa95ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:19.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.014+0000 7fa95ffff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:19.016 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:19.016 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 10 librados: watch_flush enter 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 10 librados: watch_flush exit 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 20 client.4818.objecter shutdown clearing up homeless session... 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 10 client.4818.objecter successfully canceled tick 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 msgr2=0x7fa95407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 0x7fa95407b2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa974039000 tx=0x7fa974033000 comp rx=0 tx=0).stop 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 10 monclient: shutdown 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 msgr2=0x7fa9841ed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.015+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa97000ebe0 tx=0x7fa97000c520 comp rx=0 tx=0).stop 2026-02-24T18:37:19.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 shutdown_connections 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa954078e00 0x7fa95407b2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9840c99a0 0x7fa98414cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa98415a410 0x7fa98414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 --2- 10.20.193.18:0/1761841590 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa98406ff20 0x7fa9841ed490 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 >> 10.20.193.18:0/1761841590 conn(0x7fa9840cd780 msgr2=0x7fa9840c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 shutdown_connections 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 -- 10.20.193.18:0/1761841590 wait complete. 2026-02-24T18:37:19.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:19.016+0000 7fa98a261640 1 librados: shutdown 2026-02-24T18:37:19.460 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:57.405 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.461 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:57.407 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.470 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:03:57.416 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.487 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:57.432 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.488 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:57.434 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.489 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:57.397 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.490 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:57.398 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.498 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:03:57.444 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:19.498 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:03:57.407 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:22.027 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.097+0000 7f931a624640 10 monclient: get_monmap_and_config 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.097+0000 7f931a624640 10 monclient: build_initial_monmap 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.097+0000 7f931a624640 10 monclient: monmap: 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:22.098411+0000 2026-02-24T18:37:22.099 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:22.098411+0000 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:22.100 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.097+0000 7f931a624640 1 Processor -- start 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 -- start start 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient: init 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient: _add_conns ranks=[4,1,3,2,5,0] 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f931406ff20 0x7f9314070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): picked mon.noname-b con 0x7f931406ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f93140c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f931415a5e0 0x7f931414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): picked mon.noname-a con 0x7f931415a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93140c7e90 con 0x7f93140c99a0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9314159a40 con 0x7f931415a5e0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93140c8170 con 0x7f931406ff20 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f931a624640 10 monclient(hunting): authenticate will time out at 1585.779540s 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f9318b9a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f931415a5e0 0x7f931414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.098+0000 7f9313fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f931406ff20 0x7f9314070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f931415a5e0 0x7f931414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60742/0 (socket says 10.20.193.18:60742) 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 1 -- 10.20.193.18:0/324068205 learned_addr learned my addr 10.20.193.18:0/324068205 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): get_auth_request con 0x7f931415a5e0 auth_method 0 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): get_auth_request con 0x7f931406ff20 auth_method 0 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:22.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.099+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 10 monclient(hunting): handle_auth_done global_id 4821 payload 274 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 1 -- 10.20.193.18:0/324068205 >> v2:10.20.193.32:3300/0 conn(0x7f931406ff20 msgr2=0x7f9314070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 1 --2- 10.20.193.18:0/324068205 >> v2:10.20.193.32:3300/0 conn(0x7f931406ff20 0x7f9314070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 1 -- 10.20.193.18:0/324068205 >> v1:10.20.193.32:6789/0 conn(0x7f93140c99a0 legacy=0x7f9314159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 10 monclient: _finish_hunting 0 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 1 monclient: found mon.noname-a 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.100+0000 7f9318b9a640 1 -- 10.20.193.18:0/324068205 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93140c8450 con 0x7f931415a5e0 2026-02-24T18:37:22.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.101+0000 7f9318b9a640 1 --2- 10.20.193.18:0/324068205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f931415a5e0 0x7f931414fe10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9308009e50 tx=0x7f930802f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5719f1847417fdbe server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:22.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 1 -- 10.20.193.18:0/324068205 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930802fcc0 con 0x7f931415a5e0 2026-02-24T18:37:22.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:22.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:22.113 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: dump: 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: _finish_auth 0 2026-02-24T18:37:22.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 1 -- 10.20.193.18:0/324068205 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93080373f0 con 0x7f931415a5e0 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 5 monclient: authenticate success, global_id 4821 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 1 -- 10.20.193.18:0/324068205 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930802fe60 con 0x7f931415a5e0 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f9312ffd640 10 monclient: dump: 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:22.115 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 4 set_mon_vals no callback set 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 10 monclient: shutdown 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:22.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 -- 10.20.193.18:0/324068205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f931415a5e0 msgr2=0x7f931414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 --2- 10.20.193.18:0/324068205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f931415a5e0 0x7f931414fe10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9308009e50 tx=0x7f930802f2a0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 -- 10.20.193.18:0/324068205 shutdown_connections 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 --2- 10.20.193.18:0/324068205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f931415a5e0 0x7f931414fe10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 --2- 10.20.193.18:0/324068205 >> v2:10.20.193.32:3300/0 conn(0x7f931406ff20 0x7f9314070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 -- 10.20.193.18:0/324068205 >> 10.20.193.18:0/324068205 conn(0x7f93140cd780 msgr2=0x7f93140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 -- 10.20.193.18:0/324068205 shutdown_connections 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.103+0000 7f931a624640 1 -- 10.20.193.18:0/324068205 wait complete. 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient: build_initial_monmap 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient: monmap: 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:22.117 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 librados: starting msgr at 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 librados: starting objecter 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 Processor -- start 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 -- start start 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 librados: setting wanted keys 2026-02-24T18:37:22.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 librados: calling monclient init 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient: init 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f931406ff20 0x7f93140bce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): picked mon.noname-c con 0x7f931406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): picked mon.noname-a con 0x7f93140c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f931415a5e0 0x7f93141580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): picked mon.noname-b con 0x7f931415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9314150350 con 0x7f93140c99a0 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:22.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93140c8a10 con 0x7f931415a5e0 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93141504d0 con 0x7f931406ff20 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.104+0000 7f931a624640 10 monclient(hunting): authenticate will time out at 1585.785936s 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60756/0 (socket says 10.20.193.18:60756) 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 1 -- 10.20.193.18:0/1447757170 learned_addr learned my addr 10.20.193.18:0/1447757170 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): get_auth_request con 0x7f93140c99a0 auth_method 0 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:22.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f931415a5e0 0x7f93141580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9313fff640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f931406ff20 0x7f93140bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f93137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): get_auth_request con 0x7f931415a5e0 auth_method 0 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9313fff640 10 monclient(hunting): get_auth_request con 0x7f931406ff20 auth_method 0 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9313fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9313fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:22.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9313fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.105+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f9318b9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f9313fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 10 monclient(hunting): handle_auth_done global_id 4824 payload 824 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 -- 10.20.193.18:0/1447757170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f931406ff20 msgr2=0x7f93140bce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f931406ff20 0x7f93140bce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 -- 10.20.193.18:0/1447757170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f931415a5e0 msgr2=0x7f93141580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f931415a5e0 0x7f93141580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 monclient: found mon.noname-a 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.106+0000 7f93137fe640 1 -- 10.20.193.18:0/1447757170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93141f45b0 con 0x7f93140c99a0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.108+0000 7f93137fe640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f930400c600 tx=0x7f930400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.108+0000 7f9310ff9640 10 client.?.objecter ms_handle_connect 0x7f93140c99a0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.108+0000 7f9310ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304012830 con 0x7f93140c99a0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: dump: 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:22.123 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: _finish_auth 0 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304017070 con 0x7f93140c99a0 2026-02-24T18:37:22.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304012c10 con 0x7f93140c99a0 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 5 monclient: authenticate success, global_id 4824 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 10 monclient: _renew_subs 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93141f1590 con 0x7f93140c99a0 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f93127fc640 4 set_mon_vals no callback set 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f9310ff9640 10 monclient: dump: 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.125 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 10 client.4824.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 10 monclient: _renew_subs 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93141f50b0 con 0x7f93140c99a0 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.109+0000 7f931a624640 1 librados: init done 2026-02-24T18:37:22.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.110+0000 7f931a624640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:22.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.110+0000 7f931a624640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:22.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.110+0000 7f931a624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.110+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9314150860 con 0x7f93140c99a0 2026-02-24T18:37:22.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.122+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f930401c460 con 0x7f93140c99a0 2026-02-24T18:37:22.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.122+0000 7f9310ff9640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 0x7f92e407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:22.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.122+0000 7f9313fff640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 0x7f92e407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:22.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.122+0000 7f9313fff640 10 monclient: get_auth_request con 0x7f92e4078e50 auth_method 0 2026-02-24T18:37:22.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.123+0000 7f9313fff640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 0x7f92e407b310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9300009c40 tx=0x7f9300009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:22.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.123+0000 7f9310ff9640 10 client.4824.objecter ms_handle_connect 0x7f92e4078e50 2026-02-24T18:37:22.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f930405e230 con 0x7f93140c99a0 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 10 client.4824.objecter ms_dispatch2 0x7f9314000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 3 client.4824.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 3 client.4824.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 20 client.4824.objecter dump_active .. 0 homeless 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f93040621f0 con 0x7f93140c99a0 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:22.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.227+0000 7f9310ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:22.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.327+0000 7f931a624640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:22.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.327+0000 7f931a624640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:22.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.327+0000 7f931a624640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:22.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.327+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f93141594d0 con 0x7f93140c99a0 2026-02-24T18:37:22.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.330+0000 7f9310ff9640 1 -- 10.20.193.18:0/1447757170 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f93040664f0 con 0x7f93140c99a0 2026-02-24T18:37:22.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.330+0000 7f9310ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:22.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.330+0000 7f9310ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:22.332 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:22.333 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:22.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 10 librados: watch_flush enter 2026-02-24T18:37:22.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 10 librados: watch_flush exit 2026-02-24T18:37:22.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 20 client.4824.objecter shutdown clearing up homeless session... 2026-02-24T18:37:22.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 10 client.4824.objecter successfully canceled tick 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 msgr2=0x7f92e407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 0x7f92e407b310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9300009c40 tx=0x7f9300009290 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 10 monclient: shutdown 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 msgr2=0x7f93141f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f930400c600 tx=0x7f930400cad0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 shutdown_connections 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f92e4078e50 0x7f92e407b310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f931415a5e0 0x7f93141580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f93140c99a0 0x7f93141f4070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 --2- 10.20.193.18:0/1447757170 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f931406ff20 0x7f93140bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 >> 10.20.193.18:0/1447757170 conn(0x7f93140cd780 msgr2=0x7f93140c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 shutdown_connections 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 -- 10.20.193.18:0/1447757170 wait complete. 2026-02-24T18:37:22.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:22.331+0000 7f931a624640 1 librados: shutdown 2026-02-24T18:37:25.343 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:25.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.411+0000 7f6a13586640 10 monclient: get_monmap_and_config 2026-02-24T18:37:25.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.411+0000 7f6a13586640 10 monclient: build_initial_monmap 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.411+0000 7f6a13586640 10 monclient: monmap: 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:25.412779+0000 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:25.412779+0000 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:25.416 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 Processor -- start 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 -- start start 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient: init 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient: _add_conns ranks=[3,1,2,4,0,5] 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6a0c06ff20 0x7f6a0c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-a con 0x7f6a0c06ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6a0c0c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:25.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6a0c15a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c0c7e90 con 0x7f6a0c0c99a0 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c0c8170 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c159a40 con 0x7f6a0c06ff20 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.412+0000 7f6a13586640 10 monclient(hunting): authenticate will time out at 1589.093924s 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6a0c06ff20 0x7f6a0c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f6a0c06ff20 0x7f6a0c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60758/0 (socket says 10.20.193.18:60758) 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 1 -- 10.20.193.18:0/2343696726 learned_addr learned my addr 10.20.193.18:0/2343696726 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:25.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): get_auth_request con 0x7f6a0c06ff20 auth_method 0 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:25.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.413+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:25.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 10 monclient(hunting): handle_auth_done global_id 4827 payload 274 2026-02-24T18:37:25.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 1 -- 10.20.193.18:0/2343696726 >> v1:10.20.193.110:6789/0 conn(0x7f6a0c15a5e0 legacy=0x7f6a0c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 1 -- 10.20.193.18:0/2343696726 >> v1:10.20.193.32:6789/0 conn(0x7f6a0c0c99a0 legacy=0x7f6a0c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 10 monclient: _finish_hunting 0 2026-02-24T18:37:25.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 1 monclient: found mon.noname-a 2026-02-24T18:37:25.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.414+0000 7f6a112fb640 1 -- 10.20.193.18:0/2343696726 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0c8450 con 0x7f6a0c06ff20 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.415+0000 7f6a112fb640 1 --2- 10.20.193.18:0/2343696726 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c070300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0099a0 tx=0x7f69fc02efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c101aa03a48e6de server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 1 -- 10.20.193.18:0/2343696726 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc03c070 con 0x7f6a0c06ff20 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: dump: 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.428 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: _finish_auth 0 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 1 -- 10.20.193.18:0/2343696726 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc02fc70 con 0x7f6a0c06ff20 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 5 monclient: authenticate success, global_id 4827 2026-02-24T18:37:25.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 1 -- 10.20.193.18:0/2343696726 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037660 con 0x7f6a0c06ff20 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f69fbfff640 10 monclient: dump: 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:25.430 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 4 set_mon_vals no callback set 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 10 monclient: shutdown 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 -- 10.20.193.18:0/2343696726 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c06ff20 msgr2=0x7f6a0c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 --2- 10.20.193.18:0/2343696726 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c070300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0099a0 tx=0x7f69fc02efa0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 -- 10.20.193.18:0/2343696726 shutdown_connections 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 --2- 10.20.193.18:0/2343696726 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c070300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 -- 10.20.193.18:0/2343696726 >> 10.20.193.18:0/2343696726 conn(0x7f6a0c0cd780 msgr2=0x7f6a0c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 -- 10.20.193.18:0/2343696726 shutdown_connections 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.417+0000 7f6a13586640 1 -- 10.20.193.18:0/2343696726 wait complete. 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient: build_initial_monmap 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient: monmap: 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:25.432 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 librados: starting msgr at 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 librados: starting objecter 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 Processor -- start 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 -- start start 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 librados: setting wanted keys 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 librados: calling monclient init 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient: init 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c14d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:25.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-b con 0x7f6a0c06ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a0c0c99a0 0x7f6a0c0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-c con 0x7f6a0c0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): picked mon.noname-a con 0x7f6a0c15a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c150520 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c0c8a10 con 0x7f6a0c06ff20 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:25.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1506a0 con 0x7f6a0c0c99a0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a13586640 10 monclient(hunting): authenticate will time out at 1589.099727s 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.418+0000 7f6a11afc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60760/0 (socket says 10.20.193.18:60760) 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 1 -- 10.20.193.18:0/1046763542 learned_addr learned my addr 10.20.193.18:0/1046763542 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c14d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): get_auth_request con 0x7f6a0c15a5e0 auth_method 0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a0c0c99a0 0x7f6a0c0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): get_auth_request con 0x7f6a0c06ff20 auth_method 0 2026-02-24T18:37:25.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a11afc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): get_auth_request con 0x7f6a0c0c99a0 auth_method 0 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:25.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a112fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.419+0000 7f6a10afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 10 monclient(hunting): handle_auth_done global_id 4830 payload 824 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 -- 10.20.193.18:0/1046763542 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a0c0c99a0 msgr2=0x7f6a0c0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a0c0c99a0 0x7f6a0c0c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 -- 10.20.193.18:0/1046763542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a0c06ff20 msgr2=0x7f6a0c14d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c14d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 10 monclient: _finish_hunting 0 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 monclient: found mon.noname-a 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.420+0000 7f6a11afc640 1 -- 10.20.193.18:0/1046763542 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0be8d0 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.421+0000 7f6a11afc640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800cbd0 tx=0x7f6a0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.421+0000 7f69f9ffb640 10 client.?.objecter ms_handle_connect 0x7f6a0c15a5e0 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.421+0000 7f69f9ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800ad50 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: dump: 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:25.438 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08003a70 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08003e40 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:25.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69fb7fe640 4 set_mon_vals no callback set 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f69f9ffb640 10 monclient: dump: 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:25.440 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 5 monclient: authenticate success, global_id 4830 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _renew_subs 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c0bebb0 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 client.4830.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _renew_subs 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c0bf170 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 1 librados: init done 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:25.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:25.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.423+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c070300 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.434+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6a080050d0 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.435+0000 7f69f9ffb640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 0x7f69dc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.435+0000 7f6a112fb640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 0x7f69dc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.435+0000 7f6a112fb640 10 monclient: get_auth_request con 0x7f69dc078e50 auth_method 0 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.435+0000 7f6a112fb640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 0x7f69dc07b310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009970 tx=0x7f69fc033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.435+0000 7f69f9ffb640 10 client.4830.objecter ms_handle_connect 0x7f69dc078e50 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.436+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f6a0805e690 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.436+0000 7f69f9ffb640 10 client.4830.objecter ms_dispatch2 0x7f6a0c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.436+0000 7f69f9ffb640 3 client.4830.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.436+0000 7f69f9ffb640 3 client.4830.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:25.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.436+0000 7f69f9ffb640 20 client.4830.objecter dump_active .. 0 homeless 2026-02-24T18:37:25.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.536+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6a08062650 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.536+0000 7f69f9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:25.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.536+0000 7f69f9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:25.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.638+0000 7f6a13586640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:25.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.638+0000 7f6a13586640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:25.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.638+0000 7f6a13586640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:25.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.638+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a0c14fef0 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.641+0000 7f69f9ffb640 1 -- 10.20.193.18:0/1046763542 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6a08066950 con 0x7f6a0c15a5e0 2026-02-24T18:37:25.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.641+0000 7f69f9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:25.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.641+0000 7f69f9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:25.643 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:25.644 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:25.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 10 librados: watch_flush enter 2026-02-24T18:37:25.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 10 librados: watch_flush exit 2026-02-24T18:37:25.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 20 client.4830.objecter shutdown clearing up homeless session... 2026-02-24T18:37:25.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 10 client.4830.objecter successfully canceled tick 2026-02-24T18:37:25.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 msgr2=0x7f69dc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 0x7f69dc07b310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009970 tx=0x7f69fc033000 comp rx=0 tx=0).stop 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 10 monclient: shutdown 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 msgr2=0x7f6a0c0bb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800cbd0 tx=0x7f6a0800a430 comp rx=0 tx=0).stop 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 shutdown_connections 2026-02-24T18:37:25.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f69dc078e50 0x7f69dc07b310 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a0c15a5e0 0x7f6a0c0bb7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a0c0c99a0 0x7f6a0c0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 --2- 10.20.193.18:0/1046763542 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a0c06ff20 0x7f6a0c14d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 >> 10.20.193.18:0/1046763542 conn(0x7f6a0c0cd780 msgr2=0x7f6a0c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 shutdown_connections 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 -- 10.20.193.18:0/1046763542 wait complete. 2026-02-24T18:37:25.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:25.642+0000 7f6a13586640 1 librados: shutdown 2026-02-24T18:37:28.655 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:28.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.722+0000 7f16f00ea640 10 monclient: get_monmap_and_config 2026-02-24T18:37:28.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.722+0000 7f16f00ea640 10 monclient: build_initial_monmap 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.722+0000 7f16f00ea640 10 monclient: monmap: 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:28.724795+0000 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:28.724795+0000 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:28.726 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:28.727 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:28.727 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:28.727 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.723+0000 7f16f00ea640 1 Processor -- start 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.723+0000 7f16f00ea640 1 -- start start 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.723+0000 7f16f00ea640 10 monclient: init 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient: _add_conns ranks=[1,2,3,5,4,0] 2026-02-24T18:37:28.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f16e80c90d0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f16e80c94b0 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f16e8074250 0x7f16e8074680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-a con 0x7f16e8074250 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e8073300 con 0x7f16e80c90d0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e8159440 con 0x7f16e80c94b0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f16e815ce70 con 0x7f16e8074250 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16f00ea640 10 monclient(hunting): authenticate will time out at 1592.406188s 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f16e8074250 0x7f16e8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ed65e640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f16e80c94b0 0x7f16e8158950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:44662/0 (socket says 10.20.193.18:44662) 2026-02-24T18:37:28.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ed65e640 1 -- 10.20.193.18:0/1776445063 learned_addr learned my addr 10.20.193.18:0/1776445063 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): get_auth_request con 0x7f16e8074250 auth_method 0 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.724+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685384655 0 0) 0x7f16e8159440 con 0x7f16e80c94b0 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _negotiate 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): my global_id is 4790 2026-02-24T18:37:28.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d0003620 con 0x7f16e80c94b0 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1721851761 0 0) 0x7f16e8073300 con 0x7f16e80c90d0 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _negotiate 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 10 monclient(hunting): my global_id is 4816 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16e8159440 con 0x7f16e80c90d0 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 10 monclient(hunting): handle_auth_done global_id 4833 payload 274 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 1 -- 10.20.193.18:0/1776445063 >> v1:10.20.193.110:6789/0 conn(0x7f16e80c94b0 legacy=0x7f16e8158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 1 -- 10.20.193.18:0/1776445063 >> v1:10.20.193.32:6789/0 conn(0x7f16e80c90d0 legacy=0x7f16e8072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 10 monclient: _finish_hunting 0 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 1 monclient: found mon.noname-a 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:28.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.725+0000 7f16ee660640 1 -- 10.20.193.18:0/1776445063 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e8159720 con 0x7f16e8074250 2026-02-24T18:37:28.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.726+0000 7f16ee660640 1 --2- 10.20.193.18:0/1776445063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e8074680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009d90 tx=0x7f16e402f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ddf83ebe336d46 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e403c070 con 0x7f16e8074250 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: dump: 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:28.740 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: _finish_auth 0 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: _check_auth_tickets 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e4037440 con 0x7f16e8074250 2026-02-24T18:37:28.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 1 -- 10.20.193.18:0/1776445063 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4037810 con 0x7f16e8074250 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16ece5d640 10 monclient: dump: 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:28.742 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16f00ea640 5 monclient: authenticate success, global_id 4833 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16f00ea640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.728+0000 7f16f00ea640 4 set_mon_vals no callback set 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 10 monclient: shutdown 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 -- 10.20.193.18:0/1776445063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 msgr2=0x7f16e8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1776445063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e8074680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009d90 tx=0x7f16e402f230 comp rx=0 tx=0).stop 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 -- 10.20.193.18:0/1776445063 shutdown_connections 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1776445063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e8074680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 -- 10.20.193.18:0/1776445063 >> 10.20.193.18:0/1776445063 conn(0x7f16e80cd780 msgr2=0x7f16e80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:28.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 -- 10.20.193.18:0/1776445063 shutdown_connections 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 1 -- 10.20.193.18:0/1776445063 wait complete. 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 10 monclient: build_initial_monmap 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.729+0000 7f16f00ea640 10 monclient: monmap: 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:28.744 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 librados: starting msgr at 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 librados: starting objecter 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 Processor -- start 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 -- start start 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 librados: setting wanted keys 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 librados: calling monclient init 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient: init 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-a con 0x7f16e8074250 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f16e80c90d0 0x7f16e81f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:28.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-b con 0x7f16e80c90d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16e80c94b0 0x7f16e81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): picked mon.noname-c con 0x7f16e80c94b0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8073a70 con 0x7f16e8074250 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8073540 con 0x7f16e80c90d0 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8073d50 con 0x7f16e80c94b0 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16f00ea640 10 monclient(hunting): authenticate will time out at 1592.412604s 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ee660640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16e80c94b0 0x7f16e81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:28.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60784/0 (socket says 10.20.193.18:60784) 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 1 -- 10.20.193.18:0/1313007129 learned_addr learned my addr 10.20.193.18:0/1313007129 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 10 monclient(hunting): get_auth_request con 0x7f16e8074250 auth_method 0 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.730+0000 7f16ede5f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): get_auth_request con 0x7f16e80c94b0 auth_method 0 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f16e80c90d0 0x7f16e81f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ede5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:28.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ede5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ede5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): get_auth_request con 0x7f16e80c90d0 auth_method 0 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ee660640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:28.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.731+0000 7f16ed65e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 10 monclient(hunting): handle_auth_done global_id 4836 payload 824 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 -- 10.20.193.18:0/1313007129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16e80c94b0 msgr2=0x7f16e81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16e80c94b0 0x7f16e81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 -- 10.20.193.18:0/1313007129 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f16e80c90d0 msgr2=0x7f16e81f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f16e80c90d0 0x7f16e81f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 10 monclient: _finish_hunting 0 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 monclient: found mon.noname-a 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.732+0000 7f16ede5f640 1 -- 10.20.193.18:0/1313007129 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81f7a30 con 0x7f16e8074250 2026-02-24T18:37:28.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.733+0000 7f16ede5f640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c4f0 tx=0x7f16d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.733+0000 7f16d67fc640 10 client.?.objecter ms_handle_connect 0x7f16e8074250 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.733+0000 7f16d67fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d80052d0 con 0x7f16e8074250 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: dump: 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.750 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 5 monclient: authenticate success, global_id 4836 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d8005470 con 0x7f16e8074250 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _renew_subs 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81f7d10 con 0x7f16e8074250 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:28.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8003910 con 0x7f16e8074250 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d7fff640 4 set_mon_vals no callback set 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16d67fc640 10 monclient: dump: 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:28.752 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 client.4836.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _renew_subs 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81e3e80 con 0x7f16e8074250 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 1 librados: init done 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.735+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e806e840 con 0x7f16e8074250 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.747+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f16d8003ab0 con 0x7f16e8074250 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.748+0000 7f16d67fc640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 0x7f16b807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.748+0000 7f16ed65e640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 0x7f16b807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.748+0000 7f16ed65e640 10 monclient: get_auth_request con 0x7f16b8078e50 auth_method 0 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.748+0000 7f16d67fc640 10 client.4836.objecter ms_handle_connect 0x7f16b8078e50 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.748+0000 7f16ed65e640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 0x7f16b807b310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f16dc007b60 tx=0x7f16dc007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.749+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f16d805dfe0 con 0x7f16e8074250 2026-02-24T18:37:28.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.749+0000 7f16d67fc640 10 client.4836.objecter ms_dispatch2 0x7f16e8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:28.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.749+0000 7f16d67fc640 3 client.4836.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:28.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.749+0000 7f16d67fc640 3 client.4836.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:28.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.749+0000 7f16d67fc640 20 client.4836.objecter dump_active .. 0 homeless 2026-02-24T18:37:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.851+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f16d8061fa0 con 0x7f16e8074250 2026-02-24T18:37:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.851+0000 7f16d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:28.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.851+0000 7f16d67fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:28.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.950+0000 7f16f00ea640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:28.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.950+0000 7f16f00ea640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:28.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.950+0000 7f16f00ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:28.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.950+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16e806f2d0 con 0x7f16e8074250 2026-02-24T18:37:28.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.952+0000 7f16d67fc640 1 -- 10.20.193.18:0/1313007129 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f16d80662a0 con 0x7f16e8074250 2026-02-24T18:37:28.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.952+0000 7f16d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:28.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.952+0000 7f16d67fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:28.956 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:28.956 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:28.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 10 librados: watch_flush enter 2026-02-24T18:37:28.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 10 librados: watch_flush exit 2026-02-24T18:37:28.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 20 client.4836.objecter shutdown clearing up homeless session... 2026-02-24T18:37:28.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 10 client.4836.objecter successfully canceled tick 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 msgr2=0x7f16b807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 0x7f16b807b310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f16dc007b60 tx=0x7f16dc007ab0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 10 monclient: shutdown 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 msgr2=0x7f16e81f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.953+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c4f0 tx=0x7f16d800c9c0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 shutdown_connections 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f16b8078e50 0x7f16b807b310 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f16e80c90d0 0x7f16e81f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f16e80c94b0 0x7f16e81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 --2- 10.20.193.18:0/1313007129 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f16e8074250 0x7f16e81f46a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:28.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 >> 10.20.193.18:0/1313007129 conn(0x7f16e80cd780 msgr2=0x7f16e80c66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:28.959 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 shutdown_connections 2026-02-24T18:37:28.959 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 -- 10.20.193.18:0/1313007129 wait complete. 2026-02-24T18:37:28.959 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:28.954+0000 7f16f00ea640 1 librados: shutdown 2026-02-24T18:37:31.965 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:32.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.034+0000 7f20b92f1640 10 monclient: get_monmap_and_config 2026-02-24T18:37:32.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.034+0000 7f20b92f1640 10 monclient: build_initial_monmap 2026-02-24T18:37:32.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.034+0000 7f20b92f1640 10 monclient: monmap: 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:32.035684+0000 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:32.035684+0000 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:32.037 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:32.038 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:32.038 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 Processor -- start 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 -- start start 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient: init 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient: _add_conns ranks=[4,1,0,5,3,2] 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f20b406ff20 0x7f20b4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:32.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-b con 0x7f20b406ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f20b40c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f20b415a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b40c7e90 con 0x7f20b415a5e0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b40c8170 con 0x7f20b40c99a0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f20b4159a40 con 0x7f20b406ff20 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.035+0000 7f20b92f1640 10 monclient(hunting): authenticate will time out at 1595.716885s 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.036+0000 7f20b37fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f20b415a5e0 0x7f20b414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:57202/0 (socket says 10.20.193.18:57202) 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.036+0000 7f20b37fe640 1 -- 10.20.193.18:0/218358024 learned_addr learned my addr 10.20.193.18:0/218358024 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684401600 0 0) 0x7f20b40c7e90 con 0x7f20b415a5e0 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:37:32.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 10 monclient(hunting): my global_id is 4839 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20a0003620 con 0x7f20b415a5e0 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b2ffd640 1 --2- 10.20.193.18:0/218358024 >> v2:10.20.193.32:3300/0 conn(0x7f20b406ff20 0x7f20b4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b2ffd640 10 monclient(hunting): get_auth_request con 0x7f20b406ff20 auth_method 0 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.037+0000 7f20b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.038+0000 7f20b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.038+0000 7f20b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.038+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.038+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:32.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.038+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_done global_id 4825 payload 274 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1014672454 0 0) 0x7f20a0003620 con 0x7f20b415a5e0 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 1 -- 10.20.193.18:0/218358024 >> v1:10.20.193.32:6789/0 conn(0x7f20b40c99a0 legacy=0x7f20b4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 1 -- 10.20.193.18:0/218358024 >> v1:10.20.193.18:6789/0 conn(0x7f20b415a5e0 legacy=0x7f20b414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 1 monclient: found mon.noname-b 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:32.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b2ffd640 1 -- 10.20.193.18:0/218358024 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b40c8450 con 0x7f20b406ff20 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.039+0000 7f20b1ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.040+0000 7f20b2ffd640 1 --2- 10.20.193.18:0/218358024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b4070300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20a80098d0 tx=0x7f20a802ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b328a764976b9583 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a803c070 con 0x7f20b406ff20 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: dump: 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.049 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a802f880 con 0x7f20b406ff20 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 5 monclient: authenticate success, global_id 4825 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:32.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 1 -- 10.20.193.18:0/218358024 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4003280 con 0x7f20b406ff20 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b1ffb640 10 monclient: dump: 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:32.051 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 4 set_mon_vals no callback set 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 10 monclient: shutdown 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 -- 10.20.193.18:0/218358024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 msgr2=0x7f20b4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 --2- 10.20.193.18:0/218358024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b4070300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20a80098d0 tx=0x7f20a802ee30 comp rx=0 tx=0).stop 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 -- 10.20.193.18:0/218358024 shutdown_connections 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 --2- 10.20.193.18:0/218358024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b4070300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 -- 10.20.193.18:0/218358024 >> 10.20.193.18:0/218358024 conn(0x7f20b40cd780 msgr2=0x7f20b40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 -- 10.20.193.18:0/218358024 shutdown_connections 2026-02-24T18:37:32.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.042+0000 7f20b92f1640 1 -- 10.20.193.18:0/218358024 wait complete. 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 10 monclient: build_initial_monmap 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 10 monclient: monmap: 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 librados: starting msgr at 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 librados: starting objecter 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 Processor -- start 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 -- start start 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 librados: setting wanted keys 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 1 librados: calling monclient init 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.043+0000 7f20b92f1640 10 monclient: init 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b41f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-b con 0x7f20b406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f20b40c99a0 0x7f20b41f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-c con 0x7f20b40c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:32.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): picked mon.noname-a con 0x7f20b415a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4150520 con 0x7f20b415a5e0 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20b40c8a10 con 0x7f20b406ff20 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41506a0 con 0x7f20b40c99a0 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b92f1640 10 monclient(hunting): authenticate will time out at 1595.725148s 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b41f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:32.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49872/0 (socket says 10.20.193.18:49872) 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 1 -- 10.20.193.18:0/2744361483 learned_addr learned my addr 10.20.193.18:0/2744361483 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): get_auth_request con 0x7f20b415a5e0 auth_method 0 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): get_auth_request con 0x7f20b406ff20 auth_method 0 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b27fc640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f20b40c99a0 0x7f20b41f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:32.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b27fc640 10 monclient(hunting): get_auth_request con 0x7f20b40c99a0 auth_method 0 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.044+0000 7f20b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.045+0000 7f20b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.045+0000 7f20b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.045+0000 7f20b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 10 monclient(hunting): handle_auth_done global_id 4842 payload 824 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 -- 10.20.193.18:0/2744361483 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f20b40c99a0 msgr2=0x7f20b41f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f20b40c99a0 0x7f20b41f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 -- 10.20.193.18:0/2744361483 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 msgr2=0x7f20b41f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b41f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 monclient: found mon.noname-a 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:32.058 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.046+0000 7f20b37fe640 1 -- 10.20.193.18:0/2744361483 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41f4040 con 0x7f20b415a5e0 2026-02-24T18:37:32.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.047+0000 7f20b37fe640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002f90 tx=0x7f20a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:32.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.047+0000 7f20937fe640 10 client.?.objecter ms_handle_connect 0x7f20b415a5e0 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.047+0000 7f20937fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400ad50 con 0x7f20b415a5e0 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: dump: 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.061 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: _finish_auth 0 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: _check_auth_tickets 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a4012400 con 0x7f20b415a5e0 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 5 monclient: authenticate success, global_id 4842 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _renew_subs 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41f4320 con 0x7f20b415a5e0 2026-02-24T18:37:32.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a40127f0 con 0x7f20b415a5e0 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20937fe640 10 monclient: dump: 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:32.063 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b17fa640 4 set_mon_vals no callback set 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 client.4842.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _renew_subs 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41e5010 con 0x7f20b415a5e0 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 1 librados: init done 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:32.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.049+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b41f7bc0 con 0x7f20b415a5e0 2026-02-24T18:37:32.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.061+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f20a4012990 con 0x7f20b415a5e0 2026-02-24T18:37:32.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.062+0000 7f20937fe640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 0x7f208007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:32.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.062+0000 7f20b2ffd640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 0x7f208007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:32.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.062+0000 7f20b2ffd640 10 monclient: get_auth_request con 0x7f2080078e50 auth_method 0 2026-02-24T18:37:32.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.062+0000 7f20b2ffd640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 0x7f208007b310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20a8039000 tx=0x7f20a8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:32.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.062+0000 7f20937fe640 10 client.4842.objecter ms_handle_connect 0x7f2080078e50 2026-02-24T18:37:32.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.064+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f20a405e940 con 0x7f20b415a5e0 2026-02-24T18:37:32.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.064+0000 7f20937fe640 10 client.4842.objecter ms_dispatch2 0x7f20b4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:32.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.064+0000 7f20937fe640 3 client.4842.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:32.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.064+0000 7f20937fe640 3 client.4842.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:32.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.064+0000 7f20937fe640 20 client.4842.objecter dump_active .. 0 homeless 2026-02-24T18:37:32.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.166+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f20a4062900 con 0x7f20b415a5e0 2026-02-24T18:37:32.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.166+0000 7f20937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:32.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.166+0000 7f20937fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:32.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.265+0000 7f20b92f1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:32.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.265+0000 7f20b92f1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:32.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.265+0000 7f20b92f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:32.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.265+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f20b414f3b0 con 0x7f20b415a5e0 2026-02-24T18:37:32.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.267+0000 7f20937fe640 1 -- 10.20.193.18:0/2744361483 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f20a4066c00 con 0x7f20b415a5e0 2026-02-24T18:37:32.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.267+0000 7f20937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:32.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.267+0000 7f20937fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:32.269 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:32.269 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:32.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 10 librados: watch_flush enter 2026-02-24T18:37:32.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 10 librados: watch_flush exit 2026-02-24T18:37:32.271 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 20 client.4842.objecter shutdown clearing up homeless session... 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 10 client.4842.objecter successfully canceled tick 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 msgr2=0x7f208007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 0x7f208007b310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20a8039000 tx=0x7f20a8033000 comp rx=0 tx=0).stop 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 10 monclient: shutdown 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 msgr2=0x7f20b40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002f90 tx=0x7f20a400a430 comp rx=0 tx=0).stop 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 shutdown_connections 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2080078e50 0x7f208007b310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f20b415a5e0 0x7f20b40cc7e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f20b40c99a0 0x7f20b41f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 --2- 10.20.193.18:0/2744361483 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f20b406ff20 0x7f20b41f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:32.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 >> 10.20.193.18:0/2744361483 conn(0x7f20b40cd780 msgr2=0x7f20b414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:32.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 shutdown_connections 2026-02-24T18:37:32.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 -- 10.20.193.18:0/2744361483 wait complete. 2026-02-24T18:37:32.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:32.268+0000 7f20b92f1640 1 librados: shutdown 2026-02-24T18:37:35.280 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 10 monclient: get_monmap_and_config 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 10 monclient: build_initial_monmap 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 10 monclient: monmap: 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:35.349008+0000 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:35.349008+0000 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:35.349 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 1 Processor -- start 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 1 -- start start 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.348+0000 7fbe1bcbd640 10 monclient: init 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient: _add_conns ranks=[2,5,3,1,4,0] 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbe141527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbe14152bb0 0x7fbe140c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-c con 0x7fbe14152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fbe140c5fc0 0x7fbe140c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-a con 0x7fbe140c5fc0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe1406e300 con 0x7fbe141527d0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1406f1f0 con 0x7fbe140c5fc0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140c6470 con 0x7fbe14152bb0 2026-02-24T18:37:35.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1bcbd640 10 monclient(hunting): authenticate will time out at 1599.030149s 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fbe140c5fc0 0x7fbe140c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fbe140c5fc0 0x7fbe140c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49878/0 (socket says 10.20.193.18:49878) 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 1 -- 10.20.193.18:0/3200213214 learned_addr learned my addr 10.20.193.18:0/3200213214 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): get_auth_request con 0x7fbe140c5fc0 auth_method 0 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe19231640 1 --2- 10.20.193.18:0/3200213214 >> v2:10.20.193.110:3300/0 conn(0x7fbe14152bb0 0x7fbe140c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.349+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): get_auth_request con 0x7fbe14152bb0 auth_method 0 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:35.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 10 monclient(hunting): handle_auth_done global_id 4845 payload 274 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 1 -- 10.20.193.18:0/3200213214 >> v2:10.20.193.110:3300/0 conn(0x7fbe14152bb0 msgr2=0x7fbe140c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 1 --2- 10.20.193.18:0/3200213214 >> v2:10.20.193.110:3300/0 conn(0x7fbe14152bb0 0x7fbe140c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 1 -- 10.20.193.18:0/3200213214 >> v1:10.20.193.110:6789/0 conn(0x7fbe141527d0 legacy=0x7fbe1406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 10 monclient: _finish_hunting 0 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 1 monclient: found mon.noname-a 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:35.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.350+0000 7fbe1a233640 1 -- 10.20.193.18:0/3200213214 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1406e5e0 con 0x7fbe140c5fc0 2026-02-24T18:37:35.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.351+0000 7fbe1a233640 1 --2- 10.20.193.18:0/3200213214 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe140c5fc0 0x7fbe140c8e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe140750e0 tx=0x7fbe1002f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35332bbc902cd3db server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 1 -- 10.20.193.18:0/3200213214 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1002fd70 con 0x7fbe140c5fc0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: dump: 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: _finish_auth 0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: _check_auth_tickets 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 1 -- 10.20.193.18:0/3200213214 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100373f0 con 0x7fbe140c5fc0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe1bcbd640 5 monclient: authenticate success, global_id 4845 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 1 -- 10.20.193.18:0/3200213214 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100377c0 con 0x7fbe140c5fc0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe18a30640 10 monclient: dump: 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:35.355 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe1bcbd640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:35.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.353+0000 7fbe1bcbd640 4 set_mon_vals no callback set 2026-02-24T18:37:35.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 10 monclient: shutdown 2026-02-24T18:37:35.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:35.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/3200213214 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe140c5fc0 msgr2=0x7fbe140c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/3200213214 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe140c5fc0 0x7fbe140c8e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe140750e0 tx=0x7fbe1002f270 comp rx=0 tx=0).stop 2026-02-24T18:37:35.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/3200213214 shutdown_connections 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/3200213214 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe140c5fc0 0x7fbe140c8e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/3200213214 >> v2:10.20.193.110:3300/0 conn(0x7fbe14152bb0 0x7fbe140c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/3200213214 >> 10.20.193.18:0/3200213214 conn(0x7fbe140cd780 msgr2=0x7fbe140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/3200213214 shutdown_connections 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/3200213214 wait complete. 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 10 monclient: build_initial_monmap 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 10 monclient: monmap: 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:35.361 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 librados: starting msgr at 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 librados: starting objecter 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 Processor -- start 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 -- start start 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 librados: setting wanted keys 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 1 librados: calling monclient init 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.354+0000 7fbe1bcbd640 10 monclient: init 2026-02-24T18:37:35.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbe140c5fc0 0x7fbe141f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-b con 0x7fbe140c5fc0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-c con 0x7fbe141527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 0x7fbe14157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): picked mon.noname-a con 0x7fbe14152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1406ee80 con 0x7fbe14152bb0 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1406eba0 con 0x7fbe140c5fc0 2026-02-24T18:37:35.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140c93a0 con 0x7fbe141527d0 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1bcbd640 10 monclient(hunting): authenticate will time out at 1599.036382s 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1a233640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 0x7fbe14157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57406/0 (socket says 10.20.193.18:57406) 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 1 -- 10.20.193.18:0/790350367 learned_addr learned my addr 10.20.193.18:0/790350367 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 10 monclient(hunting): get_auth_request con 0x7fbe141527d0 auth_method 0 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19231640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:35.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1a233640 10 monclient(hunting): get_auth_request con 0x7fbe14152bb0 auth_method 0 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1a233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1a233640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe1a233640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.355+0000 7fbe19a32640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbe140c5fc0 0x7fbe141f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe1a233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): get_auth_request con 0x7fbe140c5fc0 auth_method 0 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:35.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.356+0000 7fbe19a32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 10 monclient(hunting): handle_auth_done global_id 4805 payload 824 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 -- 10.20.193.18:0/790350367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbe140c5fc0 msgr2=0x7fbe141f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbe140c5fc0 0x7fbe141f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 -- 10.20.193.18:0/790350367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 msgr2=0x7fbe14157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 0x7fbe14157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 10 monclient: _finish_hunting 0 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 monclient: found mon.noname-c 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:35.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:35.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe19231640 1 -- 10.20.193.18:0/790350367 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141f57e0 con 0x7fbe141527d0 2026-02-24T18:37:35.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.357+0000 7fbe1a233640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 0x7fbe14157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:37:35.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.358+0000 7fbe19231640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e8c0 tx=0x7fbe0800ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:35.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.358+0000 7fbe027fc640 10 client.?.objecter ms_handle_connect 0x7fbe141527d0 2026-02-24T18:37:35.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.358+0000 7fbe027fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08017070 con 0x7fbe141527d0 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: dump: 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:35.369 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0800c940 con 0x7fbe141527d0 2026-02-24T18:37:35.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 5 monclient: authenticate success, global_id 4805 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _renew_subs 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141f5ac0 con 0x7fbe141527d0 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800cd30 con 0x7fbe141527d0 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe027fc640 10 monclient: dump: 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:35.371 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe03fff640 4 set_mon_vals no callback set 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 client.4805.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _renew_subs 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141f6080 con 0x7fbe141527d0 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 1 librados: init done 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:35.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:35.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:35.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.360+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe141f3860 con 0x7fbe141527d0 2026-02-24T18:37:35.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.372+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbe08024060 con 0x7fbe141527d0 2026-02-24T18:37:35.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.373+0000 7fbe027fc640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 0x7fbde407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:35.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.373+0000 7fbe19a32640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 0x7fbde407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:35.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.373+0000 7fbe19a32640 10 monclient: get_auth_request con 0x7fbde4078e00 auth_method 0 2026-02-24T18:37:35.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.373+0000 7fbe19a32640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 0x7fbde407b2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe040051f0 tx=0x7fbe04007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:35.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.373+0000 7fbe027fc640 10 client.4805.objecter ms_handle_connect 0x7fbde4078e00 2026-02-24T18:37:35.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.470+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fbe0805e170 con 0x7fbe141527d0 2026-02-24T18:37:35.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.470+0000 7fbe027fc640 10 client.4805.objecter ms_dispatch2 0x7fbe14000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:35.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.470+0000 7fbe027fc640 3 client.4805.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:35.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.470+0000 7fbe027fc640 3 client.4805.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:35.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.470+0000 7fbe027fc640 20 client.4805.objecter dump_active .. 0 homeless 2026-02-24T18:37:35.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.476+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbe08062130 con 0x7fbe141527d0 2026-02-24T18:37:35.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.476+0000 7fbe027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:35.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.476+0000 7fbe027fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:35.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.577+0000 7fbe1bcbd640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:35.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.577+0000 7fbe1bcbd640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:35.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.577+0000 7fbe1bcbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:35.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.577+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbe140c9730 con 0x7fbe141527d0 2026-02-24T18:37:35.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.580+0000 7fbe027fc640 1 -- 10.20.193.18:0/790350367 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbe08066430 con 0x7fbe141527d0 2026-02-24T18:37:35.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.580+0000 7fbe027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:35.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.580+0000 7fbe027fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:35.582 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:35.582 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 10 librados: watch_flush enter 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 10 librados: watch_flush exit 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 20 client.4805.objecter shutdown clearing up homeless session... 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 10 client.4805.objecter successfully canceled tick 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 msgr2=0x7fbde407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 0x7fbde407b2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe040051f0 tx=0x7fbe04007f30 comp rx=0 tx=0).stop 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 10 monclient: shutdown 2026-02-24T18:37:35.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 msgr2=0x7fbe141f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800e8c0 tx=0x7fbe0800ed90 comp rx=0 tx=0).stop 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 shutdown_connections 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbde4078e00 0x7fbde407b2c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbe14152bb0 0x7fbe14157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbe141527d0 0x7fbe141f4ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 --2- 10.20.193.18:0/790350367 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbe140c5fc0 0x7fbe141f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 >> 10.20.193.18:0/790350367 conn(0x7fbe140cd780 msgr2=0x7fbe140c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 shutdown_connections 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 -- 10.20.193.18:0/790350367 wait complete. 2026-02-24T18:37:35.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:35.581+0000 7fbe1bcbd640 1 librados: shutdown 2026-02-24T18:37:38.592 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 10 monclient: get_monmap_and_config 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 10 monclient: build_initial_monmap 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 10 monclient: monmap: 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:38.662 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:38.661048+0000 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:38.661048+0000 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 1 Processor -- start 2026-02-24T18:37:38.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 1 -- start start 2026-02-24T18:37:38.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.660+0000 7f850c038640 10 monclient: init 2026-02-24T18:37:38.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:38.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient: _add_conns ranks=[2,4,5,3,0,1] 2026-02-24T18:37:38.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f850406ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:38.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f85040c99a0 0x7f8504070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): picked mon.noname-b con 0x7f85040c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f850415a5e0 0x7f850414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): picked mon.noname-c con 0x7f850415a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85040c7e90 con 0x7f850406ff20 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8504159a40 con 0x7f85040c99a0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f85040c8170 con 0x7f850415a5e0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850c038640 10 monclient(hunting): authenticate will time out at 1602.342188s 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f85040c99a0 0x7f8504070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f850a5ae640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f850415a5e0 0x7f850414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f85040c99a0 0x7f8504070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:56780/0 (socket says 10.20.193.18:56780) 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 1 -- 10.20.193.18:0/496926072 learned_addr learned my addr 10.20.193.18:0/496926072 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 10 monclient(hunting): get_auth_request con 0x7f85040c99a0 auth_method 0 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.661+0000 7f85095ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): get_auth_request con 0x7f850415a5e0 auth_method 0 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:38.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.662+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 10 monclient(hunting): handle_auth_done global_id 4834 payload 274 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 1 -- 10.20.193.18:0/496926072 >> v2:10.20.193.110:3300/0 conn(0x7f850415a5e0 msgr2=0x7f850414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 1 --2- 10.20.193.18:0/496926072 >> v2:10.20.193.110:3300/0 conn(0x7f850415a5e0 0x7f850414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 1 -- 10.20.193.18:0/496926072 >> v1:10.20.193.110:6789/0 conn(0x7f850406ff20 legacy=0x7f8504159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 10 monclient: _finish_hunting 0 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 1 monclient: found mon.noname-b 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.663+0000 7f85095ac640 1 -- 10.20.193.18:0/496926072 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85040c8450 con 0x7f85040c99a0 2026-02-24T18:37:38.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.664+0000 7f85095ac640 1 --2- 10.20.193.18:0/496926072 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f85040c99a0 0x7f8504070300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009970 tx=0x7f84f802eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45992b437fa2ffc server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.665+0000 7f8508dab640 1 -- 10.20.193.18:0/496926072 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f803c070 con 0x7f85040c99a0 2026-02-24T18:37:38.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: dump: 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:38.677 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: _finish_auth 0 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: _check_auth_tickets 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 1 -- 10.20.193.18:0/496926072 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f802fbc0 con 0x7f85040c99a0 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 5 monclient: authenticate success, global_id 4834 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 1 -- 10.20.193.18:0/496926072 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037440 con 0x7f85040c99a0 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:38.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 4 set_mon_vals no callback set 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f8508dab640 10 monclient: dump: 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:38.679 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 10 monclient: shutdown 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 -- 10.20.193.18:0/496926072 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f85040c99a0 msgr2=0x7f8504070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 --2- 10.20.193.18:0/496926072 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f85040c99a0 0x7f8504070300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009970 tx=0x7f84f802eef0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 -- 10.20.193.18:0/496926072 shutdown_connections 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 --2- 10.20.193.18:0/496926072 >> v2:10.20.193.110:3300/0 conn(0x7f850415a5e0 0x7f850414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 --2- 10.20.193.18:0/496926072 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f85040c99a0 0x7f8504070300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 -- 10.20.193.18:0/496926072 >> 10.20.193.18:0/496926072 conn(0x7f85040cd780 msgr2=0x7f85040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 -- 10.20.193.18:0/496926072 shutdown_connections 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.666+0000 7f850c038640 1 -- 10.20.193.18:0/496926072 wait complete. 2026-02-24T18:37:38.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient: build_initial_monmap 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient: monmap: 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 librados: starting msgr at 2026-02-24T18:37:38.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 librados: starting objecter 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 Processor -- start 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 -- start start 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 librados: setting wanted keys 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 librados: calling monclient init 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient: init 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): picked mon.noname-c con 0x7f850406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f85040c99a0 0x7f85041f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): picked mon.noname-a con 0x7f85040c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:38.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f850415a5e0 0x7f85040c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): picked mon.noname-b con 0x7f850415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8504150350 con 0x7f85040c99a0 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85040c8a10 con 0x7f850415a5e0 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85041504d0 con 0x7f850406ff20 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.667+0000 7f850c038640 10 monclient(hunting): authenticate will time out at 1602.348730s 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f85040c99a0 0x7f85041f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.683 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57422/0 (socket says 10.20.193.18:57422) 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 1 -- 10.20.193.18:0/348317058 learned_addr learned my addr 10.20.193.18:0/348317058 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): get_auth_request con 0x7f850406ff20 auth_method 0 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): get_auth_request con 0x7f85040c99a0 auth_method 0 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f8509dad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:38.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.668+0000 7f85095ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f850415a5e0 0x7f85040c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): get_auth_request con 0x7f850415a5e0 auth_method 0 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f850a5ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 10 monclient(hunting): handle_auth_done global_id 4814 payload 824 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 -- 10.20.193.18:0/348317058 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f850415a5e0 msgr2=0x7f85040c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f850415a5e0 0x7f85040c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 -- 10.20.193.18:0/348317058 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f85040c99a0 msgr2=0x7f85041f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f85040c99a0 0x7f85041f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 10 monclient: _finish_hunting 0 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 monclient: found mon.noname-c 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.669+0000 7f8509dad640 1 -- 10.20.193.18:0/348317058 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85041f2640 con 0x7f850406ff20 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.671+0000 7f8509dad640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c580 tx=0x7f84f400ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.671+0000 7f84f27fc640 10 client.?.objecter ms_handle_connect 0x7f850406ff20 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.671+0000 7f84f27fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4017070 con 0x7f850406ff20 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:38.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: dump: 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f40052d0 con 0x7f850406ff20 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f40036b0 con 0x7f850406ff20 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 5 monclient: authenticate success, global_id 4814 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f3fff640 4 set_mon_vals no callback set 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f84f27fc640 10 monclient: dump: 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:38.688 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 10 monclient: _renew_subs 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041f2920 con 0x7f850406ff20 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 10 client.4814.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 10 monclient: _renew_subs 2026-02-24T18:37:38.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85040c2bd0 con 0x7f850406ff20 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.672+0000 7f850c038640 1 librados: init done 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.673+0000 7f850c038640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.673+0000 7f850c038640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.673+0000 7f850c038640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:38.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.673+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504150860 con 0x7f850406ff20 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.684+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f84f4003850 con 0x7f850406ff20 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.685+0000 7f84f27fc640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 0x7f84d407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.685+0000 7f85095ac640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 0x7f84d407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.685+0000 7f85095ac640 10 monclient: get_auth_request con 0x7f84d4078e50 auth_method 0 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.685+0000 7f85095ac640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 0x7f84d407b310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85040709c0 tx=0x7f84f8039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.685+0000 7f84f27fc640 10 client.4814.objecter ms_handle_connect 0x7f84d4078e50 2026-02-24T18:37:38.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.686+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f84f405d3f0 con 0x7f850406ff20 2026-02-24T18:37:38.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.686+0000 7f84f27fc640 10 client.4814.objecter ms_dispatch2 0x7f8504000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:38.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.687+0000 7f84f27fc640 3 client.4814.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:38.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.687+0000 7f84f27fc640 3 client.4814.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:38.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.687+0000 7f84f27fc640 20 client.4814.objecter dump_active .. 0 homeless 2026-02-24T18:37:38.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.789+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f84f4021020 con 0x7f850406ff20 2026-02-24T18:37:38.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.789+0000 7f84f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:38.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.789+0000 7f84f27fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:38.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.892+0000 7f850c038640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:38.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.892+0000 7f850c038640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:38.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.892+0000 7f850c038640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:38.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.892+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8504159800 con 0x7f850406ff20 2026-02-24T18:37:38.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.895+0000 7f84f27fc640 1 -- 10.20.193.18:0/348317058 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f84f40613b0 con 0x7f850406ff20 2026-02-24T18:37:38.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.895+0000 7f84f27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:38.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.895+0000 7f84f27fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:38.896 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:38.897 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 10 librados: watch_flush enter 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 10 librados: watch_flush exit 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 20 client.4814.objecter shutdown clearing up homeless session... 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 10 client.4814.objecter successfully canceled tick 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 msgr2=0x7f84d407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 0x7f84d407b310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85040709c0 tx=0x7f84f8039040 comp rx=0 tx=0).stop 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 10 monclient: shutdown 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:38.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 msgr2=0x7f85041f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c580 tx=0x7f84f400ca50 comp rx=0 tx=0).stop 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 shutdown_connections 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f84d4078e50 0x7f84d407b310 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f850415a5e0 0x7f85040c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f85040c99a0 0x7f85041f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 --2- 10.20.193.18:0/348317058 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f850406ff20 0x7f85041f5c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 >> 10.20.193.18:0/348317058 conn(0x7f85040cd780 msgr2=0x7f85040c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 shutdown_connections 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 -- 10.20.193.18:0/348317058 wait complete. 2026-02-24T18:37:38.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:38.896+0000 7f850c038640 1 librados: shutdown 2026-02-24T18:37:41.908 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:41.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 10 monclient: get_monmap_and_config 2026-02-24T18:37:41.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 10 monclient: build_initial_monmap 2026-02-24T18:37:41.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 10 monclient: monmap: 2026-02-24T18:37:41.978 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:41.978 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:41.977129+0000 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:41.977129+0000 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 1 Processor -- start 2026-02-24T18:37:41.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 1 -- start start 2026-02-24T18:37:41.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.976+0000 7f3f87b96640 10 monclient: init 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient: _add_conns ranks=[2,5,3,0,1,4] 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3f801527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f3f80152bb0 0x7f3f800c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-c con 0x7f3f80152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3f800c5fc0 0x7f3f800c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-a con 0x7f3f800c5fc0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8006e300 con 0x7f3f801527d0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8006f1f0 con 0x7f3f800c5fc0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3f800c6470 con 0x7f3f80152bb0 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:41.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f87b96640 10 monclient(hunting): authenticate will time out at 1605.658297s 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8610c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3f800c5fc0 0x7f3f800c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8590b640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f3f801527d0 0x7f3f8006da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:45574/0 (socket says 10.20.193.18:45574) 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8590b640 1 -- 10.20.193.18:0/2587651566 learned_addr learned my addr 10.20.193.18:0/2587651566 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8610c640 10 monclient(hunting): get_auth_request con 0x7f3f800c5fc0 auth_method 0 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8610c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8610c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.977+0000 7f3f8610c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.978+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.978+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.978+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 10 monclient(hunting): handle_auth_done global_id 4854 payload 274 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 1 -- 10.20.193.18:0/2587651566 >> v2:10.20.193.110:3300/0 conn(0x7f3f80152bb0 msgr2=0x7f3f800c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 1 --2- 10.20.193.18:0/2587651566 >> v2:10.20.193.110:3300/0 conn(0x7f3f80152bb0 0x7f3f800c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 1 -- 10.20.193.18:0/2587651566 >> v1:10.20.193.110:6789/0 conn(0x7f3f801527d0 legacy=0x7f3f8006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 10 monclient: _finish_hunting 0 2026-02-24T18:37:41.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 1 monclient: found mon.noname-a 2026-02-24T18:37:41.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:41.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:41.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.979+0000 7f3f8610c640 1 -- 10.20.193.18:0/2587651566 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8006e5e0 con 0x7f3f800c5fc0 2026-02-24T18:37:41.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.980+0000 7f3f8610c640 1 --2- 10.20.193.18:0/2587651566 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f800c8e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f800750e0 tx=0x7f3f7c02f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eb9291c01c8333 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:41.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 1 -- 10.20.193.18:0/2587651566 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c03c070 con 0x7f3f800c5fc0 2026-02-24T18:37:41.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:41.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:41.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: dump: 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: _finish_auth 0 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: _check_auth_tickets 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 1 -- 10.20.193.18:0/2587651566 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7c037440 con 0x7f3f800c5fc0 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 5 monclient: authenticate success, global_id 4854 2026-02-24T18:37:41.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 1 -- 10.20.193.18:0/2587651566 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c037810 con 0x7f3f800c5fc0 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f84909640 10 monclient: dump: 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 4 set_mon_vals no callback set 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 10 monclient: shutdown 2026-02-24T18:37:41.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 -- 10.20.193.18:0/2587651566 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 msgr2=0x7f3f800c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 --2- 10.20.193.18:0/2587651566 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f800c8e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f800750e0 tx=0x7f3f7c02f1c0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 -- 10.20.193.18:0/2587651566 shutdown_connections 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 --2- 10.20.193.18:0/2587651566 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f800c8e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 --2- 10.20.193.18:0/2587651566 >> v2:10.20.193.110:3300/0 conn(0x7f3f80152bb0 0x7f3f800c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 -- 10.20.193.18:0/2587651566 >> 10.20.193.18:0/2587651566 conn(0x7f3f800cd780 msgr2=0x7f3f800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 -- 10.20.193.18:0/2587651566 shutdown_connections 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.982+0000 7f3f87b96640 1 -- 10.20.193.18:0/2587651566 wait complete. 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient: build_initial_monmap 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient: monmap: 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:41.990 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 librados: starting msgr at 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 librados: starting objecter 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 Processor -- start 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 -- start start 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 librados: setting wanted keys 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 librados: calling monclient init 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient: init 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-a con 0x7f3f800c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f801527d0 0x7f3f80147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-b con 0x7f3f801527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f80152bb0 0x7f3f801f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): picked mon.noname-c con 0x7f3f80152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8006ee80 con 0x7f3f800c5fc0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8006eba0 con 0x7f3f801527d0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f800c93a0 con 0x7f3f80152bb0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.983+0000 7f3f87b96640 10 monclient(hunting): authenticate will time out at 1605.664881s 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38086/0 (socket says 10.20.193.18:38086) 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 1 -- 10.20.193.18:0/1979620137 learned_addr learned my addr 10.20.193.18:0/1979620137 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): get_auth_request con 0x7f3f800c5fc0 auth_method 0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f801527d0 0x7f3f80147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f80152bb0 0x7f3f801f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8590b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): get_auth_request con 0x7f3f801527d0 auth_method 0 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): get_auth_request con 0x7f3f80152bb0 auth_method 0 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8510a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.984+0000 7f3f8610c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 10 monclient(hunting): handle_auth_done global_id 4857 payload 824 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 -- 10.20.193.18:0/1979620137 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f80152bb0 msgr2=0x7f3f801f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f80152bb0 0x7f3f801f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 -- 10.20.193.18:0/1979620137 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f801527d0 msgr2=0x7f3f80147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f801527d0 0x7f3f80147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 10 monclient: _finish_hunting 0 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 monclient: found mon.noname-a 2026-02-24T18:37:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.985+0000 7f3f8590b640 1 -- 10.20.193.18:0/1979620137 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f80073280 con 0x7f3f800c5fc0 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.986+0000 7f3f8590b640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c730 tx=0x7f3f7000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.986+0000 7f3f6e7fc640 10 client.?.objecter ms_handle_connect 0x7f3f800c5fc0 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.986+0000 7f3f6e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70017070 con 0x7f3f800c5fc0 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: dump: 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.995 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70005b90 con 0x7f3f800c5fc0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 5 monclient: authenticate success, global_id 4857 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007070 con 0x7f3f800c5fc0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 10 monclient: _renew_subs 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f80073560 con 0x7f3f800c5fc0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6ffff640 4 set_mon_vals no callback set 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f6e7fc640 10 monclient: dump: 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:41.996 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 10 client.4857.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 10 monclient: _renew_subs 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f80073b20 con 0x7f3f800c5fc0 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.988+0000 7f3f87b96640 1 librados: init done 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.989+0000 7f3f87b96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.989+0000 7f3f87b96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.989+0000 7f3f87b96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:41.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:41.989+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f800c9730 con 0x7f3f800c5fc0 2026-02-24T18:37:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.001+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3f70007210 con 0x7f3f800c5fc0 2026-02-24T18:37:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.001+0000 7f3f6e7fc640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 0x7f3f5007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.002+0000 7f3f8510a640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 0x7f3f5007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.002+0000 7f3f8510a640 10 monclient: get_auth_request con 0x7f3f50078e50 auth_method 0 2026-02-24T18:37:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.002+0000 7f3f8510a640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 0x7f3f5007b310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f801488f0 tx=0x7f3f74009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.002+0000 7f3f6e7fc640 10 client.4857.objecter ms_handle_connect 0x7f3f50078e50 2026-02-24T18:37:42.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.004+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f3f7005d520 con 0x7f3f800c5fc0 2026-02-24T18:37:42.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.004+0000 7f3f6e7fc640 10 client.4857.objecter ms_dispatch2 0x7f3f80000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:42.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.004+0000 7f3f6e7fc640 3 client.4857.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:42.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.004+0000 7f3f6e7fc640 3 client.4857.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:42.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.004+0000 7f3f6e7fc640 20 client.4857.objecter dump_active .. 0 homeless 2026-02-24T18:37:42.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.106+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3f70021020 con 0x7f3f800c5fc0 2026-02-24T18:37:42.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.106+0000 7f3f6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:42.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.106+0000 7f3f6e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:42.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.207+0000 7f3f87b96640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:42.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.207+0000 7f3f87b96640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:42.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.207+0000 7f3f87b96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:42.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.207+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3f800c50f0 con 0x7f3f800c5fc0 2026-02-24T18:37:42.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.210+0000 7f3f6e7fc640 1 -- 10.20.193.18:0/1979620137 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3f700614e0 con 0x7f3f800c5fc0 2026-02-24T18:37:42.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.210+0000 7f3f6e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:42.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.210+0000 7f3f6e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:42.212 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:42.212 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 10 librados: watch_flush enter 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 10 librados: watch_flush exit 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 20 client.4857.objecter shutdown clearing up homeless session... 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 10 client.4857.objecter successfully canceled tick 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 msgr2=0x7f3f5007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 0x7f3f5007b310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f801488f0 tx=0x7f3f74009190 comp rx=0 tx=0).stop 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 10 monclient: shutdown 2026-02-24T18:37:42.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 msgr2=0x7f3f80072c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c730 tx=0x7f3f7000cc00 comp rx=0 tx=0).stop 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 shutdown_connections 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3f50078e50 0x7f3f5007b310 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3f801527d0 0x7f3f80147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3f80152bb0 0x7f3f801f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 --2- 10.20.193.18:0/1979620137 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3f800c5fc0 0x7f3f80072c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 >> 10.20.193.18:0/1979620137 conn(0x7f3f800cd780 msgr2=0x7f3f800ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 shutdown_connections 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 -- 10.20.193.18:0/1979620137 wait complete. 2026-02-24T18:37:42.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:42.211+0000 7f3f87b96640 1 librados: shutdown 2026-02-24T18:37:45.223 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 10 monclient: get_monmap_and_config 2026-02-24T18:37:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 10 monclient: build_initial_monmap 2026-02-24T18:37:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 10 monclient: monmap: 2026-02-24T18:37:45.292 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:45.292 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:45.292137+0000 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:45.292137+0000 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 1 Processor -- start 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 1 -- start start 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.291+0000 7fc95f409640 10 monclient: init 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient: _add_conns ranks=[1,4,0,2,5,3] 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc95806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc9580c99a0 0x7fc958070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-b con 0x7fc9580c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc95815a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9580c7e90 con 0x7fc95815a5e0 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9580c8170 con 0x7fc95806ff20 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc958159a40 con 0x7fc9580c99a0 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95f409640 10 monclient(hunting): authenticate will time out at 1608.973314s 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95d97f640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fc95815a5e0 0x7fc95814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:40258/0 (socket says 10.20.193.18:40258) 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95d97f640 1 -- 10.20.193.18:0/3867384756 learned_addr learned my addr 10.20.193.18:0/3867384756 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.292+0000 7fc95c97d640 1 --2- 10.20.193.18:0/3867384756 >> v2:10.20.193.32:3300/0 conn(0x7fc9580c99a0 0x7fc958070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): get_auth_request con 0x7fc9580c99a0 auth_method 0 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.293+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 1 -- 10.20.193.18:0/3867384756 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757531701 0 0) 0x7fc9580c7e90 con 0x7fc95815a5e0 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 10 monclient(hunting): _negotiate 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 10 monclient(hunting): my global_id is 4860 2026-02-24T18:37:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc947fff640 1 -- 10.20.193.18:0/3867384756 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc940003620 con 0x7fc95815a5e0 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 10 monclient(hunting): handle_auth_done global_id 4843 payload 274 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 1 -- 10.20.193.18:0/3867384756 >> v1:10.20.193.32:6789/0 conn(0x7fc95806ff20 legacy=0x7fc958159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 1 -- 10.20.193.18:0/3867384756 >> v1:10.20.193.18:6789/0 conn(0x7fc95815a5e0 legacy=0x7fc95814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 10 monclient: _finish_hunting 0 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 1 monclient: found mon.noname-b 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.294+0000 7fc95c97d640 1 -- 10.20.193.18:0/3867384756 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9580c8450 con 0x7fc9580c99a0 2026-02-24T18:37:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.296+0000 7fc95c97d640 1 --2- 10.20.193.18:0/3867384756 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9580c99a0 0x7fc958070300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0098d0 tx=0x7fc94c02ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=153b067d57627035 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 1 -- 10.20.193.18:0/3867384756 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03c070 con 0x7fc9580c99a0 2026-02-24T18:37:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: dump: 2026-02-24T18:37:45.300 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: _finish_auth 0 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 1 -- 10.20.193.18:0/3867384756 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c02fb00 con 0x7fc9580c99a0 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 5 monclient: authenticate success, global_id 4843 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 1 -- 10.20.193.18:0/3867384756 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c037440 con 0x7fc9580c99a0 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 4 set_mon_vals no callback set 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc947fff640 10 monclient: dump: 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 10 monclient: shutdown 2026-02-24T18:37:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 -- 10.20.193.18:0/3867384756 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9580c99a0 msgr2=0x7fc958070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 --2- 10.20.193.18:0/3867384756 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9580c99a0 0x7fc958070300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0098d0 tx=0x7fc94c02ee80 comp rx=0 tx=0).stop 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 -- 10.20.193.18:0/3867384756 shutdown_connections 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 --2- 10.20.193.18:0/3867384756 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9580c99a0 0x7fc958070300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 -- 10.20.193.18:0/3867384756 >> 10.20.193.18:0/3867384756 conn(0x7fc9580cd780 msgr2=0x7fc9580cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 -- 10.20.193.18:0/3867384756 shutdown_connections 2026-02-24T18:37:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.298+0000 7fc95f409640 1 -- 10.20.193.18:0/3867384756 wait complete. 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 10 monclient: build_initial_monmap 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 10 monclient: monmap: 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 librados: starting msgr at 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 librados: starting objecter 2026-02-24T18:37:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 Processor -- start 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 -- start start 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 librados: setting wanted keys 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 1 librados: calling monclient init 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.299+0000 7fc95f409640 10 monclient: init 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 0x7fc958149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-b con 0x7fc95806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9580c99a0 0x7fc9581f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-c con 0x7fc9580c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 0x7fc9581f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): picked mon.noname-a con 0x7fc95815a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc958150520 con 0x7fc95815a5e0 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9580c8a10 con 0x7fc95806ff20 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9581506a0 con 0x7fc9580c99a0 2026-02-24T18:37:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95f409640 10 monclient(hunting): authenticate will time out at 1608.981216s 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 0x7fc9581f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 0x7fc958149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 0x7fc9581f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38088/0 (socket says 10.20.193.18:38088) 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 1 -- 10.20.193.18:0/2262345552 learned_addr learned my addr 10.20.193.18:0/2262345552 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 10 monclient(hunting): get_auth_request con 0x7fc95815a5e0 auth_method 0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d97f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95c97d640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9580c99a0 0x7fc9581f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): get_auth_request con 0x7fc95806ff20 auth_method 0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95c97d640 10 monclient(hunting): get_auth_request con 0x7fc9580c99a0 auth_method 0 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95c97d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95c97d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95c97d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.300+0000 7fc95d17e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95d97f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95d97f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95d97f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.301+0000 7fc95c97d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 10 monclient(hunting): handle_auth_done global_id 4846 payload 824 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 -- 10.20.193.18:0/2262345552 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9580c99a0 msgr2=0x7fc9581f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9580c99a0 0x7fc9581f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 -- 10.20.193.18:0/2262345552 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 msgr2=0x7fc9581f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 0x7fc9581f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 10 monclient: _finish_hunting 0 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 monclient: found mon.noname-b 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.302+0000 7fc95d17e640 1 -- 10.20.193.18:0/2262345552 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9581f71d0 con 0x7fc95806ff20 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.303+0000 7fc95d17e640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 0x7fc958149c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b550 tx=0x7fc94800ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.303+0000 7fc945ffb640 10 client.?.objecter ms_handle_connect 0x7fc95806ff20 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.303+0000 7fc945ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948004500 con 0x7fc95806ff20 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 10 monclient: dump: 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc945ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 5 monclient: authenticate success, global_id 4846 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 10 monclient: _renew_subs 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9581f74b0 con 0x7fc95806ff20 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 10 client.4846.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 10 monclient: _renew_subs 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9581f7a70 con 0x7fc95806ff20 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.304+0000 7fc95f409640 1 librados: init done 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc948015070 con 0x7fc95806ff20 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948002980 con 0x7fc95806ff20 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc945ffb640 10 monclient: dump: 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc9477fe640 4 set_mon_vals no callback set 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc95f409640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc95f409640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc95f409640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.305+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc958070300 con 0x7fc95806ff20 2026-02-24T18:37:45.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.316+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc948002b20 con 0x7fc95806ff20 2026-02-24T18:37:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.316+0000 7fc945ffb640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 0x7fc92807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.316+0000 7fc95c97d640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 0x7fc92807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.317+0000 7fc95c97d640 10 monclient: get_auth_request con 0x7fc928078e00 auth_method 0 2026-02-24T18:37:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.317+0000 7fc945ffb640 10 client.4846.objecter ms_handle_connect 0x7fc928078e00 2026-02-24T18:37:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.317+0000 7fc95c97d640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 0x7fc92807b2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc94c039000 tx=0x7fc94c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:45.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.318+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fc948097a90 con 0x7fc95806ff20 2026-02-24T18:37:45.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.318+0000 7fc945ffb640 10 client.4846.objecter ms_dispatch2 0x7fc958000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:45.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.318+0000 7fc945ffb640 3 client.4846.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:45.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.318+0000 7fc945ffb640 3 client.4846.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:45.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.318+0000 7fc945ffb640 20 client.4846.objecter dump_active .. 0 homeless 2026-02-24T18:37:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.420+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc948097e20 con 0x7fc95806ff20 2026-02-24T18:37:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.420+0000 7fc945ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.420+0000 7fc945ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:45.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.520+0000 7fc95f409640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:45.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.520+0000 7fc95f409640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:45.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.520+0000 7fc95f409640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:45.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.520+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc95814f5b0 con 0x7fc95806ff20 2026-02-24T18:37:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc945ffb640 1 -- 10.20.193.18:0/2262345552 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc948062260 con 0x7fc95806ff20 2026-02-24T18:37:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc945ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc945ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:45.526 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:45.526 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 10 librados: watch_flush enter 2026-02-24T18:37:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 10 librados: watch_flush exit 2026-02-24T18:37:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 20 client.4846.objecter shutdown clearing up homeless session... 2026-02-24T18:37:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 10 client.4846.objecter successfully canceled tick 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 msgr2=0x7fc92807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 0x7fc92807b2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc94c039000 tx=0x7fc94c033000 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 10 monclient: shutdown 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 msgr2=0x7fc958149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 0x7fc958149c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b550 tx=0x7fc94800ba20 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 shutdown_connections 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc928078e00 0x7fc92807b2c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc95815a5e0 0x7fc9581f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9580c99a0 0x7fc9581f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 --2- 10.20.193.18:0/2262345552 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc95806ff20 0x7fc958149c70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:45.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 >> 10.20.193.18:0/2262345552 conn(0x7fc9580cd780 msgr2=0x7fc9580c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:45.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 shutdown_connections 2026-02-24T18:37:45.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.525+0000 7fc95f409640 1 -- 10.20.193.18:0/2262345552 wait complete. 2026-02-24T18:37:45.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:45.526+0000 7fc95f409640 1 librados: shutdown 2026-02-24T18:37:48.537 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 10 monclient: get_monmap_and_config 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 10 monclient: build_initial_monmap 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 10 monclient: monmap: 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:48.607143+0000 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:48.607143+0000 2026-02-24T18:37:48.609 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 1 Processor -- start 2026-02-24T18:37:48.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 1 -- start start 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.606+0000 7f74ce674640 10 monclient: init 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient: _add_conns ranks=[4,5,2,0,1,3] 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 0x7f74c8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-b con 0x7f74c806ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f74c80c99a0 0x7f74c8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-c con 0x7f74c80c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f74c815a410 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c815a860 con 0x7f74c815a410 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f74c8159a40 con 0x7f74c806ff20 2026-02-24T18:37:48.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:48.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f74c80c7e90 con 0x7f74c80c99a0 2026-02-24T18:37:48.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:48.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74ce674640 10 monclient(hunting): authenticate will time out at 1612.288276s 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74c77fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f74c80c99a0 0x7f74c8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.607+0000 7f74c7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 0x7f74c8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 0x7f74c8070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:34956/0 (socket says 10.20.193.18:34956) 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 1 -- 10.20.193.18:0/3808477022 learned_addr learned my addr 10.20.193.18:0/3808477022 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): get_auth_request con 0x7f74c806ff20 auth_method 0 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:48.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): get_auth_request con 0x7f74c80c99a0 auth_method 0 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:48.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.608+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 10 monclient(hunting): handle_auth_done global_id 4826 payload 274 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 1 -- 10.20.193.18:0/3808477022 >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 msgr2=0x7f74c8070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 1 --2- 10.20.193.18:0/3808477022 >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 0x7f74c8070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 1 -- 10.20.193.18:0/3808477022 >> v1:10.20.193.110:6789/0 conn(0x7f74c815a410 legacy=0x7f74c814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 1 monclient: found mon.noname-c 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.609+0000 7f74c77fe640 1 -- 10.20.193.18:0/3808477022 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80c8170 con 0x7f74c80c99a0 2026-02-24T18:37:48.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.611+0000 7f74c77fe640 1 --2- 10.20.193.18:0/3808477022 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c8159ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74b80098a0 tx=0x7f74b802ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cef695a601f2996 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:48.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 1 -- 10.20.193.18:0/3808477022 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b803c070 con 0x7f74c80c99a0 2026-02-24T18:37:48.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:48.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: dump: 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 1 -- 10.20.193.18:0/3808477022 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b802fa40 con 0x7f74c80c99a0 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 5 monclient: authenticate success, global_id 4826 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 1 -- 10.20.193.18:0/3808477022 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b802fe10 con 0x7f74c80c99a0 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 4 set_mon_vals no callback set 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:48.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74c6ffd640 10 monclient: dump: 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 10 monclient: shutdown 2026-02-24T18:37:48.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 -- 10.20.193.18:0/3808477022 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 msgr2=0x7f74c8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 --2- 10.20.193.18:0/3808477022 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c8159ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74b80098a0 tx=0x7f74b802ee10 comp rx=0 tx=0).stop 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 -- 10.20.193.18:0/3808477022 shutdown_connections 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 --2- 10.20.193.18:0/3808477022 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c8159ed0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 --2- 10.20.193.18:0/3808477022 >> v2:10.20.193.32:3300/0 conn(0x7f74c806ff20 0x7f74c8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 -- 10.20.193.18:0/3808477022 >> 10.20.193.18:0/3808477022 conn(0x7f74c80cd780 msgr2=0x7f74c80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 -- 10.20.193.18:0/3808477022 shutdown_connections 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.613+0000 7f74ce674640 1 -- 10.20.193.18:0/3808477022 wait complete. 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient: build_initial_monmap 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient: monmap: 2026-02-24T18:37:48.627 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 librados: starting msgr at 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 librados: starting objecter 2026-02-24T18:37:48.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 Processor -- start 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 -- start start 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 librados: setting wanted keys 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 librados: calling monclient init 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient: init 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f74c806ff20 0x7f74c80c0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-b con 0x7f74c806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:48.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-c con 0x7f74c80c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f74c815a410 0x7f74c80becc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): picked mon.noname-a con 0x7f74c815a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80c8a10 con 0x7f74c815a410 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80c8730 con 0x7f74c806ff20 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8150520 con 0x7f74c80c99a0 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.614+0000 7f74ce674640 10 monclient(hunting): authenticate will time out at 1612.295769s 2026-02-24T18:37:48.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f74c806ff20 0x7f74c80c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f74c815a410 0x7f74c80becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:35420/0 (socket says 10.20.193.18:35420) 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 1 -- 10.20.193.18:0/4076408458 learned_addr learned my addr 10.20.193.18:0/4076408458 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): get_auth_request con 0x7f74c80c99a0 auth_method 0 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): get_auth_request con 0x7f74c806ff20 auth_method 0 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:48.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): get_auth_request con 0x7f74c815a410 auth_method 0 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:48.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.615+0000 7f74ccbea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 10 monclient(hunting): handle_auth_done global_id 4829 payload 824 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 -- 10.20.193.18:0/4076408458 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f74c806ff20 msgr2=0x7f74c80c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f74c806ff20 0x7f74c80c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 -- 10.20.193.18:0/4076408458 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f74c815a410 msgr2=0x7f74c80becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f74c815a410 0x7f74c80becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 monclient: found mon.noname-c 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.616+0000 7f74c77fe640 1 -- 10.20.193.18:0/4076408458 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80bf200 con 0x7f74c80c99a0 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.617+0000 7f74c77fe640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74b80099d0 tx=0x7f74b80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.617+0000 7f74c4ff9640 10 client.?.objecter ms_handle_connect 0x7f74c80c99a0 2026-02-24T18:37:48.633 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.617+0000 7f74c4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b803c040 con 0x7f74c80c99a0 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: dump: 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:48.634 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b8047070 con 0x7f74c80c99a0 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80428b0 con 0x7f74c80c99a0 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c4ff9640 10 monclient: dump: 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:48.635 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74c67fc640 4 set_mon_vals no callback set 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 5 monclient: authenticate success, global_id 4829 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 10 monclient: _renew_subs 2026-02-24T18:37:48.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c80bdc20 con 0x7f74c80c99a0 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 10 client.4829.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 10 monclient: _renew_subs 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c80be1e0 con 0x7f74c80c99a0 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.619+0000 7f74ce674640 1 librados: init done 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.620+0000 7f74ce674640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.620+0000 7f74ce674640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.620+0000 7f74ce674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.620+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c81f24c0 con 0x7f74c80c99a0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.631+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f74b8042a50 con 0x7f74c80c99a0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.631+0000 7f74c4ff9640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 0x7f749807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.631+0000 7f74c7fff640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 0x7f749807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.631+0000 7f74c7fff640 10 monclient: get_auth_request con 0x7f7498078e00 auth_method 0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.632+0000 7f74c7fff640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 0x7f749807b2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74c814d5c0 tx=0x7f74b4009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.632+0000 7f74c4ff9640 10 client.4829.objecter ms_handle_connect 0x7f7498078e00 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.633+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f74b802f900 con 0x7f74c80c99a0 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.633+0000 7f74c4ff9640 10 client.4829.objecter ms_dispatch2 0x7f74c8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:48.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.633+0000 7f74c4ff9640 3 client.4829.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:48.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.633+0000 7f74c4ff9640 3 client.4829.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:48.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.633+0000 7f74c4ff9640 20 client.4829.objecter dump_active .. 0 homeless 2026-02-24T18:37:48.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.732+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f74b80879f0 con 0x7f74c80c99a0 2026-02-24T18:37:48.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.732+0000 7f74c4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:48.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.732+0000 7f74c4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:48.832 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.831+0000 7f74ce674640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:48.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.831+0000 7f74ce674640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:48.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.831+0000 7f74ce674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:48.833 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.831+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f74c81508b0 con 0x7f74c80c99a0 2026-02-24T18:37:48.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74c4ff9640 1 -- 10.20.193.18:0/4076408458 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f74b808bcf0 con 0x7f74c80c99a0 2026-02-24T18:37:48.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74c4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:48.835 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74c4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:48.836 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:48.836 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:48.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 10 librados: watch_flush enter 2026-02-24T18:37:48.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 10 librados: watch_flush exit 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 20 client.4829.objecter shutdown clearing up homeless session... 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 10 client.4829.objecter successfully canceled tick 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 msgr2=0x7f749807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 0x7f749807b2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74c814d5c0 tx=0x7f74b4009340 comp rx=0 tx=0).stop 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 10 monclient: shutdown 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 msgr2=0x7f74c81f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74b80099d0 tx=0x7f74b80047c0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 shutdown_connections 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7498078e00 0x7f749807b2c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f74c815a410 0x7f74c80becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f74c80c99a0 0x7f74c81f1f80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 --2- 10.20.193.18:0/4076408458 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f74c806ff20 0x7f74c80c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 >> 10.20.193.18:0/4076408458 conn(0x7f74c80cd780 msgr2=0x7f74c80c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 shutdown_connections 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 -- 10.20.193.18:0/4076408458 wait complete. 2026-02-24T18:37:48.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:48.834+0000 7f74ce674640 1 librados: shutdown 2026-02-24T18:37:51.846 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:51.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.912+0000 7fc9cb555640 10 monclient: get_monmap_and_config 2026-02-24T18:37:51.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.912+0000 7fc9cb555640 10 monclient: build_initial_monmap 2026-02-24T18:37:51.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.912+0000 7fc9cb555640 10 monclient: monmap: 2026-02-24T18:37:51.914 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:51.913433+0000 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:51.913433+0000 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:51.915 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:51.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 Processor -- start 2026-02-24T18:37:51.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 -- start start 2026-02-24T18:37:51.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient: init 2026-02-24T18:37:51.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:51.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient: _add_conns ranks=[3,4,1,5,0,2] 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc9c41527d0 0x7fc9c406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-a con 0x7fc9c41527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc9c4152bb0 0x7fc9c40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-b con 0x7fc9c4152bb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc9c40c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c406e300 con 0x7fc9c40c5fc0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c406f1f0 con 0x7fc9c41527d0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c40c6470 con 0x7fc9c4152bb0 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:51.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.913+0000 7fc9cb555640 10 monclient(hunting): authenticate will time out at 1615.594655s 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc9c4152bb0 0x7fc9c40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fc9c4152bb0 0x7fc9c40c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:49302/0 (socket says 10.20.193.18:49302) 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/1573719760 learned_addr learned my addr 10.20.193.18:0/1573719760 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): get_auth_request con 0x7fc9c4152bb0 auth_method 0 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c92ca640 1 --2- 10.20.193.18:0/1573719760 >> v2:10.20.193.18:3300/0 conn(0x7fc9c41527d0 0x7fc9c406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.914+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_done global_id 4855 payload 274 2026-02-24T18:37:51.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/1573719760 >> v2:10.20.193.18:3300/0 conn(0x7fc9c41527d0 msgr2=0x7fc9c406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 1 --2- 10.20.193.18:0/1573719760 >> v2:10.20.193.18:3300/0 conn(0x7fc9c41527d0 0x7fc9c406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/1573719760 >> v1:10.20.193.32:6789/0 conn(0x7fc9c40c5fc0 legacy=0x7fc9c40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 10 monclient: _finish_hunting 0 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 1 monclient: found mon.noname-b 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.915+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/1573719760 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c406e5e0 con 0x7fc9c4152bb0 2026-02-24T18:37:51.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.916+0000 7fc9c8ac9640 1 --2- 10.20.193.18:0/1573719760 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40c5a80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40750e0 tx=0x7fc9b802eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb3b899f96c95c5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 1 -- 10.20.193.18:0/1573719760 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b803c070 con 0x7fc9c4152bb0 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: dump: 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:51.926 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: _finish_auth 0 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: _check_auth_tickets 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:51.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9cb555640 5 monclient: authenticate success, global_id 4855 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 1 -- 10.20.193.18:0/1573719760 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b802fb40 con 0x7fc9c4152bb0 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9cb555640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9cb555640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 1 -- 10.20.193.18:0/1573719760 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8037440 con 0x7fc9c4152bb0 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9cb555640 4 set_mon_vals no callback set 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.919+0000 7fc9b3fff640 10 monclient: dump: 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:51.928 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 10 monclient: shutdown 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 -- 10.20.193.18:0/1573719760 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c4152bb0 msgr2=0x7fc9c40c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 --2- 10.20.193.18:0/1573719760 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40c5a80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40750e0 tx=0x7fc9b802eed0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 -- 10.20.193.18:0/1573719760 shutdown_connections 2026-02-24T18:37:51.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 --2- 10.20.193.18:0/1573719760 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40c5a80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 --2- 10.20.193.18:0/1573719760 >> v2:10.20.193.18:3300/0 conn(0x7fc9c41527d0 0x7fc9c406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 -- 10.20.193.18:0/1573719760 >> 10.20.193.18:0/1573719760 conn(0x7fc9c40cd780 msgr2=0x7fc9c40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 -- 10.20.193.18:0/1573719760 shutdown_connections 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 1 -- 10.20.193.18:0/1573719760 wait complete. 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 10 monclient: build_initial_monmap 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.920+0000 7fc9cb555640 10 monclient: monmap: 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:51.930 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 librados: starting msgr at 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 librados: starting objecter 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 Processor -- start 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 -- start start 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 librados: setting wanted keys 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 librados: calling monclient init 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient: init 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:51.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9c40c5fc0 0x7fc9c41efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-c con 0x7fc9c40c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-b con 0x7fc9c41527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): picked mon.noname-a con 0x7fc9c4152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c406ee80 con 0x7fc9c4152bb0 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c406eba0 con 0x7fc9c41527d0 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c406dfa0 con 0x7fc9c40c5fc0 2026-02-24T18:37:51.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.921+0000 7fc9cb555640 10 monclient(hunting): authenticate will time out at 1615.602859s 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:49304/0 (socket says 10.20.193.18:49304) 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/2217651390 learned_addr learned my addr 10.20.193.18:0/2217651390 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): get_auth_request con 0x7fc9c41527d0 auth_method 0 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9c40c5fc0 0x7fc9c41efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:51.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): get_auth_request con 0x7fc9c40c5fc0 auth_method 0 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.922+0000 7fc9c92ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 10 monclient(hunting): handle_auth_done global_id 4858 payload 824 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/2217651390 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9c40c5fc0 msgr2=0x7fc9c41efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9c40c5fc0 0x7fc9c41efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/2217651390 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc9c4152bb0 msgr2=0x7fc9c40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:51.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 10 monclient: _finish_hunting 0 2026-02-24T18:37:51.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 monclient: found mon.noname-b 2026-02-24T18:37:51.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:51.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:37:51.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.924+0000 7fc9c8ac9640 1 -- 10.20.193.18:0/2217651390 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c41f0390 con 0x7fc9c41527d0 2026-02-24T18:37:51.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.925+0000 7fc9c8ac9640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9b802f3e0 tx=0x7fc9b80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.925+0000 7fc9b1ffb640 10 client.?.objecter ms_handle_connect 0x7fc9c41527d0 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.925+0000 7fc9b1ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8047070 con 0x7fc9c41527d0 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 10 monclient: dump: 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:51.937 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 5 monclient: authenticate success, global_id 4858 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _renew_subs 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:51.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c40c0250 con 0x7fc9c41527d0 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 client.4858.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _renew_subs 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c40c0810 con 0x7fc9c41527d0 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 1 librados: init done 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:51.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:51.926+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c406da60 con 0x7fc9c41527d0 2026-02-24T18:37:52.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b802fac0 con 0x7fc9c41527d0 2026-02-24T18:37:52.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:52.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8051060 con 0x7fc9c41527d0 2026-02-24T18:37:52.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 10 monclient: dump: 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:52.033 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:52.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b37fe640 4 set_mon_vals no callback set 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc9b8054080 con 0x7fc9c41527d0 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.031+0000 7fc9b1ffb640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 0x7fc99407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.032+0000 7fc9c92ca640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 0x7fc99407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.032+0000 7fc9c92ca640 10 monclient: get_auth_request con 0x7fc994078e00 auth_method 0 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.032+0000 7fc9c92ca640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 0x7fc99407b2c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4006fd0 tx=0x7fc9b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:52.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.032+0000 7fc9b1ffb640 10 client.4858.objecter ms_handle_connect 0x7fc994078e00 2026-02-24T18:37:52.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.033+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fc9b8082d70 con 0x7fc9c41527d0 2026-02-24T18:37:52.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.033+0000 7fc9b1ffb640 10 client.4858.objecter ms_dispatch2 0x7fc9c4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:52.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.033+0000 7fc9b1ffb640 3 client.4858.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:52.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.033+0000 7fc9b1ffb640 3 client.4858.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:52.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.033+0000 7fc9b1ffb640 20 client.4858.objecter dump_active .. 0 homeless 2026-02-24T18:37:52.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.041+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc9b8086d30 con 0x7fc9c41527d0 2026-02-24T18:37:52.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.041+0000 7fc9b1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:52.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.041+0000 7fc9b1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:52.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.138+0000 7fc9cb555640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:52.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.138+0000 7fc9cb555640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:52.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.138+0000 7fc9cb555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:37:52.140 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.138+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc9c41586d0 con 0x7fc9c41527d0 2026-02-24T18:37:52.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.141+0000 7fc9b1ffb640 1 -- 10.20.193.18:0/2217651390 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc9b808b030 con 0x7fc9c41527d0 2026-02-24T18:37:52.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.141+0000 7fc9b1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:52.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.141+0000 7fc9b1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:52.142 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:52.142 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 10 librados: watch_flush enter 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 10 librados: watch_flush exit 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 20 client.4858.objecter shutdown clearing up homeless session... 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 10 client.4858.objecter successfully canceled tick 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 msgr2=0x7fc99407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 0x7fc99407b2c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4006fd0 tx=0x7fc9b4008040 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 10 monclient: shutdown 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 msgr2=0x7fc9c4156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9b802f3e0 tx=0x7fc9b80047c0 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 shutdown_connections 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc994078e00 0x7fc99407b2c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc9c41527d0 0x7fc9c4156ff0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc9c4152bb0 0x7fc9c40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 --2- 10.20.193.18:0/2217651390 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc9c40c5fc0 0x7fc9c41efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 >> 10.20.193.18:0/2217651390 conn(0x7fc9c40cd780 msgr2=0x7fc9c40cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 shutdown_connections 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 -- 10.20.193.18:0/2217651390 wait complete. 2026-02-24T18:37:52.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:52.142+0000 7fc9cb555640 1 librados: shutdown 2026-02-24T18:37:55.154 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:55.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 10 monclient: get_monmap_and_config 2026-02-24T18:37:55.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 10 monclient: build_initial_monmap 2026-02-24T18:37:55.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 10 monclient: monmap: 2026-02-24T18:37:55.222 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:55.222107+0000 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:55.222107+0000 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:55.223 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 1 Processor -- start 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 1 -- start start 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.220+0000 7f815e7ff640 10 monclient: init 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient: _add_conns ranks=[4,5,0,3,1,2] 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f815806ff20 0x7f8158070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-b con 0x7f815806ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f81580c99a0 0x7f8158159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-c con 0x7f81580c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f815815a410 addr v1:10.20.193.18:6789/0 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815815a860 con 0x7f815815a410 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8158159a40 con 0x7f815806ff20 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81580c7e90 con 0x7f81580c99a0 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815e7ff640 10 monclient(hunting): authenticate will time out at 1618.903212s 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815cd75640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f815815a410 0x7f815814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58808/0 (socket says 10.20.193.18:58808) 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f815cd75640 1 -- 10.20.193.18:0/457652707 learned_addr learned my addr 10.20.193.18:0/457652707 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f8157fff640 1 --2- 10.20.193.18:0/457652707 >> v2:10.20.193.32:3300/0 conn(0x7f815806ff20 0x7f8158070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f81577fe640 1 --2- 10.20.193.18:0/457652707 >> v2:10.20.193.110:3300/0 conn(0x7f81580c99a0 0x7f8158159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f8157fff640 10 monclient(hunting): get_auth_request con 0x7f815806ff20 auth_method 0 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f8157fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f8157fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f8157fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f81577fe640 10 monclient(hunting): get_auth_request con 0x7f81580c99a0 auth_method 0 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f81577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f81577fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.221+0000 7f81577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 1 -- 10.20.193.18:0/457652707 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1737035713 0 0) 0x7f815815a860 con 0x7f815815a410 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 10 monclient(hunting): my global_id is 4869 2026-02-24T18:37:55.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.222+0000 7f8156ffd640 1 -- 10.20.193.18:0/457652707 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f813c003620 con 0x7f815815a410 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 10 monclient(hunting): handle_auth_done global_id 4835 payload 274 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 1 -- 10.20.193.18:0/457652707 >> v2:10.20.193.32:3300/0 conn(0x7f815806ff20 msgr2=0x7f8158070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 1 --2- 10.20.193.18:0/457652707 >> v2:10.20.193.32:3300/0 conn(0x7f815806ff20 0x7f8158070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 1 -- 10.20.193.18:0/457652707 >> v1:10.20.193.18:6789/0 conn(0x7f815815a410 legacy=0x7f815814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 10 monclient: _finish_hunting 0 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 1 monclient: found mon.noname-c 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.223+0000 7f81577fe640 1 -- 10.20.193.18:0/457652707 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81580c8170 con 0x7f81580c99a0 2026-02-24T18:37:55.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.224+0000 7f81577fe640 1 --2- 10.20.193.18:0/457652707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f81580c99a0 0x7f8158159ed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81480098a0 tx=0x7f814802ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42dd60401c3fddaf server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:55.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 1 -- 10.20.193.18:0/457652707 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814803c070 con 0x7f81580c99a0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: dump: 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: _finish_auth 0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 1 -- 10.20.193.18:0/457652707 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814802fae0 con 0x7f81580c99a0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 5 monclient: authenticate success, global_id 4835 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 1 -- 10.20.193.18:0/457652707 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148037440 con 0x7f81580c99a0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 4 set_mon_vals no callback set 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f8156ffd640 10 monclient: dump: 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:55.229 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 10 monclient: shutdown 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 -- 10.20.193.18:0/457652707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f81580c99a0 msgr2=0x7f8158159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 --2- 10.20.193.18:0/457652707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f81580c99a0 0x7f8158159ed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81480098a0 tx=0x7f814802ee10 comp rx=0 tx=0).stop 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 -- 10.20.193.18:0/457652707 shutdown_connections 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 --2- 10.20.193.18:0/457652707 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f81580c99a0 0x7f8158159ed0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 --2- 10.20.193.18:0/457652707 >> v2:10.20.193.32:3300/0 conn(0x7f815806ff20 0x7f8158070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 -- 10.20.193.18:0/457652707 >> 10.20.193.18:0/457652707 conn(0x7f81580cd780 msgr2=0x7f81580cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 -- 10.20.193.18:0/457652707 shutdown_connections 2026-02-24T18:37:55.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.226+0000 7f815e7ff640 1 -- 10.20.193.18:0/457652707 wait complete. 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient: build_initial_monmap 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient: monmap: 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:55.231 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 librados: starting msgr at 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 librados: starting objecter 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 Processor -- start 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 -- start start 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 librados: setting wanted keys 2026-02-24T18:37:55.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 librados: calling monclient init 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient: init 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-c con 0x7f815806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 0x7f81580bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-a con 0x7f81580c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f815815a410 0x7f81581f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): picked mon.noname-b con 0x7f815815a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:55.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f81580c8a10 con 0x7f81580c99a0 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81580c8730 con 0x7f815815a410 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8158150520 con 0x7f815806ff20 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.227+0000 7f815e7ff640 10 monclient(hunting): authenticate will time out at 1618.909817s 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 0x7f81580bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f815815a410 0x7f81581f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:59278/0 (socket says 10.20.193.18:59278) 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 0x7f81580bcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38338/0 (socket says 10.20.193.18:38338) 2026-02-24T18:37:55.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 1 -- 10.20.193.18:0/3323638168 learned_addr learned my addr 10.20.193.18:0/3323638168 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): get_auth_request con 0x7f815806ff20 auth_method 0 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): get_auth_request con 0x7f81580c99a0 auth_method 0 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): get_auth_request con 0x7f815815a410 auth_method 0 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:55.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f8157fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f81577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:55.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.228+0000 7f815cd75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:55.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 10 monclient(hunting): handle_auth_done global_id 4838 payload 824 2026-02-24T18:37:55.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 -- 10.20.193.18:0/3323638168 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f815815a410 msgr2=0x7f81581f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f815815a410 0x7f81581f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 -- 10.20.193.18:0/3323638168 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 msgr2=0x7f81580bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 0x7f81580bcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 10 monclient: _finish_hunting 0 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 monclient: found mon.noname-c 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.229+0000 7f8157fff640 1 -- 10.20.193.18:0/3323638168 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81580ba680 con 0x7f815806ff20 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.230+0000 7f8157fff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f814400ebe0 tx=0x7f814400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.230+0000 7f8154ff9640 10 client.?.objecter ms_handle_connect 0x7f815806ff20 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.230+0000 7f8154ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144017070 con 0x7f815806ff20 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:37:55.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: dump: 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: _finish_auth 0 2026-02-24T18:37:55.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814400cc40 con 0x7f815806ff20 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144013470 con 0x7f815806ff20 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 5 monclient: authenticate success, global_id 4838 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 10 monclient: _renew_subs 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f81567fc640 4 set_mon_vals no callback set 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81580ba960 con 0x7f815806ff20 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f8154ff9640 10 monclient: dump: 2026-02-24T18:37:55.242 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 10 client.4838.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 10 monclient: _renew_subs 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81580baf20 con 0x7f815806ff20 2026-02-24T18:37:55.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.232+0000 7f815e7ff640 1 librados: init done 2026-02-24T18:37:55.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.233+0000 7f815e7ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:55.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.233+0000 7f815e7ff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:55.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.233+0000 7f815e7ff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.233+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8158070300 con 0x7f815806ff20 2026-02-24T18:37:55.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.244+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8144013610 con 0x7f815806ff20 2026-02-24T18:37:55.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.245+0000 7f8154ff9640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 0x7f812807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:55.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.245+0000 7f81577fe640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 0x7f812807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:55.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.245+0000 7f81577fe640 10 monclient: get_auth_request con 0x7f8128078e50 auth_method 0 2026-02-24T18:37:55.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.245+0000 7f81577fe640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 0x7f812807b310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8148039000 tx=0x7f8148033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:55.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.245+0000 7f8154ff9640 10 client.4838.objecter ms_handle_connect 0x7f8128078e50 2026-02-24T18:37:55.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.246+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f814405e560 con 0x7f815806ff20 2026-02-24T18:37:55.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.246+0000 7f8154ff9640 10 client.4838.objecter ms_dispatch2 0x7f8158000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:55.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.246+0000 7f8154ff9640 3 client.4838.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:55.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.246+0000 7f8154ff9640 3 client.4838.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:55.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.246+0000 7f8154ff9640 20 client.4838.objecter dump_active .. 0 homeless 2026-02-24T18:37:55.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.346+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8144062520 con 0x7f815806ff20 2026-02-24T18:37:55.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.346+0000 7f8154ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:55.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.346+0000 7f8154ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:55.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.446+0000 7f815e7ff640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:55.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.446+0000 7f815e7ff640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:55.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.446+0000 7f815e7ff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:37:55.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.446+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81580cdd70 con 0x7f815806ff20 2026-02-24T18:37:55.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f8154ff9640 1 -- 10.20.193.18:0/3323638168 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8144066820 con 0x7f815806ff20 2026-02-24T18:37:55.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f8154ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:55.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f8154ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:55.452 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:55.452 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:55.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 10 librados: watch_flush enter 2026-02-24T18:37:55.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 10 librados: watch_flush exit 2026-02-24T18:37:55.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 20 client.4838.objecter shutdown clearing up homeless session... 2026-02-24T18:37:55.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 10 client.4838.objecter successfully canceled tick 2026-02-24T18:37:55.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 msgr2=0x7f812807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 0x7f812807b310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8148039000 tx=0x7f8148033000 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 10 monclient: shutdown 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 msgr2=0x7f8158073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f814400ebe0 tx=0x7f814400c520 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 shutdown_connections 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8128078e50 0x7f812807b310 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f815815a410 0x7f81581f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f81580c99a0 0x7f81580bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 --2- 10.20.193.18:0/3323638168 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f815806ff20 0x7f8158073dc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:55.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 >> 10.20.193.18:0/3323638168 conn(0x7f81580cd780 msgr2=0x7f815814f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:55.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 shutdown_connections 2026-02-24T18:37:55.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 -- 10.20.193.18:0/3323638168 wait complete. 2026-02-24T18:37:55.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:55.450+0000 7f815e7ff640 1 librados: shutdown 2026-02-24T18:37:58.463 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:58.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.531+0000 7f3900024640 10 monclient: get_monmap_and_config 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.531+0000 7f3900024640 10 monclient: build_initial_monmap 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.531+0000 7f3900024640 10 monclient: monmap: 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:37:58.532476+0000 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:37:58.532476+0000 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.534 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 Processor -- start 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 -- start start 2026-02-24T18:37:58.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient: init 2026-02-24T18:37:58.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:58.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient: _add_conns ranks=[2,3,5,0,1,4] 2026-02-24T18:37:58.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f38f806ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f38f80c99a0 0x7f38f8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): picked mon.noname-a con 0x7f38f80c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f38f815a5e0 0x7f38f814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): picked mon.noname-c con 0x7f38f815a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f80c7e90 con 0x7f38f806ff20 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f38f8159a40 con 0x7f38f80c99a0 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f38f80c8170 con 0x7f38f815a5e0 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f3900024640 10 monclient(hunting): authenticate will time out at 1622.213639s 2026-02-24T18:37:58.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.532+0000 7f38fd598640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f38f80c99a0 0x7f38f8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f38f80c99a0 0x7f38f8070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38346/0 (socket says 10.20.193.18:38346) 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 1 -- 10.20.193.18:0/1793920162 learned_addr learned my addr 10.20.193.18:0/1793920162 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): get_auth_request con 0x7f38f80c99a0 auth_method 0 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.533+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fe59a640 1 --2- 10.20.193.18:0/1793920162 >> v2:10.20.193.110:3300/0 conn(0x7f38f815a5e0 0x7f38f814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 10 monclient(hunting): handle_auth_done global_id 4875 payload 274 2026-02-24T18:37:58.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 1 -- 10.20.193.18:0/1793920162 >> v2:10.20.193.110:3300/0 conn(0x7f38f815a5e0 msgr2=0x7f38f814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 1 --2- 10.20.193.18:0/1793920162 >> v2:10.20.193.110:3300/0 conn(0x7f38f815a5e0 0x7f38f814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 1 -- 10.20.193.18:0/1793920162 >> v1:10.20.193.110:6789/0 conn(0x7f38f806ff20 legacy=0x7f38f8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 10 monclient: _finish_hunting 0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 1 monclient: found mon.noname-a 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.534+0000 7f38fd598640 1 -- 10.20.193.18:0/1793920162 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f80c8450 con 0x7f38f80c99a0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.535+0000 7f38fd598640 1 --2- 10.20.193.18:0/1793920162 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f8070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009970 tx=0x7f38ec02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bf28e2af2e54a1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 1 -- 10.20.193.18:0/1793920162 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec03c070 con 0x7f38f80c99a0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:58.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 10 monclient: dump: 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:58.542 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 10 monclient: _finish_auth 0 2026-02-24T18:37:58.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 10 monclient: _check_auth_tickets 2026-02-24T18:37:58.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f38fcd97640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:58.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f3900024640 5 monclient: authenticate success, global_id 4875 2026-02-24T18:37:58.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.536+0000 7f3900024640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 1 -- 10.20.193.18:0/1793920162 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec02fbc0 con 0x7f38f80c99a0 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 1 -- 10.20.193.18:0/1793920162 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037440 con 0x7f38f80c99a0 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f38fcd97640 10 monclient: dump: 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:58.549 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 10 monclient: get_monmap_and_config success 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 4 set_mon_vals no callback set 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 10 monclient: shutdown 2026-02-24T18:37:58.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 -- 10.20.193.18:0/1793920162 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 msgr2=0x7f38f8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 --2- 10.20.193.18:0/1793920162 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f8070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009970 tx=0x7f38ec02eef0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 -- 10.20.193.18:0/1793920162 shutdown_connections 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 --2- 10.20.193.18:0/1793920162 >> v2:10.20.193.110:3300/0 conn(0x7f38f815a5e0 0x7f38f814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 --2- 10.20.193.18:0/1793920162 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f8070300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 -- 10.20.193.18:0/1793920162 >> 10.20.193.18:0/1793920162 conn(0x7f38f80cd780 msgr2=0x7f38f80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 -- 10.20.193.18:0/1793920162 shutdown_connections 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.537+0000 7f3900024640 1 -- 10.20.193.18:0/1793920162 wait complete. 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 10 monclient: build_initial_monmap 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 10 monclient: monmap: 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:37:58.551 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 librados: starting msgr at 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 librados: starting objecter 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 Processor -- start 2026-02-24T18:37:58.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 -- start start 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 librados: setting wanted keys 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 librados: calling monclient init 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 10 monclient: init 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 10 monclient: _reopen_session rank -1 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.538+0000 7f3900024640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f38f806ff20 0x7f38f81f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): picked mon.noname-b con 0x7f38f806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): picked mon.noname-a con 0x7f38f80c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f38f815a5e0 0x7f38f80c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): picked mon.noname-c con 0x7f38f815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:58.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8150350 con 0x7f38f80c99a0 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38f80c8a10 con 0x7f38f806ff20 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): start opening mon connection 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38f81504d0 con 0x7f38f815a5e0 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): _renew_subs 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f3900024640 10 monclient(hunting): authenticate will time out at 1622.220047s 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38352/0 (socket says 10.20.193.18:38352) 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 1 -- 10.20.193.18:0/1050045549 learned_addr learned my addr 10.20.193.18:0/1050045549 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): get_auth_request con 0x7f38f80c99a0 auth_method 0 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:58.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f38f815a5e0 0x7f38f80c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fd598640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): get_auth_request con 0x7f38f815a5e0 auth_method 0 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fdd99640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f38f806ff20 0x7f38f81f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:58.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.539+0000 7f38fe59a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): get_auth_request con 0x7f38f806ff20 auth_method 0 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.540+0000 7f38fdd99640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 10 monclient(hunting): handle_auth_done global_id 4878 payload 824 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 -- 10.20.193.18:0/1050045549 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f38f815a5e0 msgr2=0x7f38f80c0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f38f815a5e0 0x7f38f80c0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 -- 10.20.193.18:0/1050045549 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f38f806ff20 msgr2=0x7f38f81f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f38f806ff20 0x7f38f81f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 10 monclient: _finish_hunting 0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 monclient: found mon.noname-a 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.541+0000 7f38fd598640 1 -- 10.20.193.18:0/1050045549 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f81f40d0 con 0x7f38f80c99a0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.542+0000 7f38fd598640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009aa0 tx=0x7f38ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.542+0000 7f38e67fc640 10 client.?.objecter ms_handle_connect 0x7f38f80c99a0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.542+0000 7f38e67fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec03c040 con 0x7f38f80c99a0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: dump: 2026-02-24T18:37:58.557 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: _finish_auth 0 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: _check_auth_tickets 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:37:58.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 5 monclient: authenticate success, global_id 4878 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _renew_subs 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f81f43b0 con 0x7f38f80c99a0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 client.4878.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _renew_subs 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f80c1170 con 0x7f38f80c99a0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 1 librados: init done 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f8150860 con 0x7f38f80c99a0 2026-02-24T18:37:58.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec02fba0 con 0x7f38f80c99a0 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec042830 con 0x7f38f80c99a0 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e7fff640 4 set_mon_vals no callback set 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.543+0000 7f38e67fc640 10 monclient: dump: 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:37:58.560 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:37:58.561 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:37:58.561 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:37:58.561 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:37:58.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.650+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f38ec0429d0 con 0x7f38f80c99a0 2026-02-24T18:37:58.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.651+0000 7f38e67fc640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 0x7f38c807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:37:58.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.651+0000 7f38fdd99640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 0x7f38c807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:37:58.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.651+0000 7f38fdd99640 10 monclient: get_auth_request con 0x7f38c8078e00 auth_method 0 2026-02-24T18:37:58.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.652+0000 7f38fdd99640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 0x7f38c807b2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002770 tx=0x7f38e8007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:37:58.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.652+0000 7f38e67fc640 10 client.4878.objecter ms_handle_connect 0x7f38c8078e00 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f38ec042cf0 con 0x7f38f80c99a0 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 10 client.4878.objecter ms_dispatch2 0x7f38f8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 3 client.4878.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 3 client.4878.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 20 client.4878.objecter dump_active .. 0 homeless 2026-02-24T18:37:58.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f38ec087920 con 0x7f38f80c99a0 2026-02-24T18:37:58.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:37:58.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.659+0000 7f38e67fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:37:58.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.761+0000 7f3900024640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:37:58.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.761+0000 7f3900024640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:58.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.761+0000 7f3900024640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:37:58.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.761+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f38f814f2d0 con 0x7f38f80c99a0 2026-02-24T18:37:58.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.764+0000 7f38e67fc640 1 -- 10.20.193.18:0/1050045549 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f38ec08bc20 con 0x7f38f80c99a0 2026-02-24T18:37:58.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.764+0000 7f38e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:37:58.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.764+0000 7f38e67fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:37:58.766 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:37:58.766 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 10 librados: watch_flush enter 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 10 librados: watch_flush exit 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 20 client.4878.objecter shutdown clearing up homeless session... 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 10 client.4878.objecter successfully canceled tick 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 msgr2=0x7f38c807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 0x7f38c807b2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002770 tx=0x7f38e8007ef0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 10 monclient: shutdown 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:37:58.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 msgr2=0x7f38f81f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009aa0 tx=0x7f38ec0047c0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 shutdown_connections 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f38c8078e00 0x7f38c807b2c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f38f815a5e0 0x7f38f80c0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f38f80c99a0 0x7f38f81f3830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 --2- 10.20.193.18:0/1050045549 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f38f806ff20 0x7f38f81f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 >> 10.20.193.18:0/1050045549 conn(0x7f38f80cd780 msgr2=0x7f38f8157840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 shutdown_connections 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 -- 10.20.193.18:0/1050045549 wait complete. 2026-02-24T18:37:58.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:37:58.765+0000 7f3900024640 1 librados: shutdown 2026-02-24T18:37:59.516 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:37.462 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.518 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:37.463 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.528 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:37.474 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.546 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:37.454 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.546 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:37.492 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.547 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:37.455 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.547 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:37.493 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.557 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:37.503 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:37:59.560 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:37.468 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:01.776 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:01.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.845+0000 7f1ec1cda640 10 monclient: get_monmap_and_config 2026-02-24T18:38:01.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.845+0000 7f1ec1cda640 10 monclient: build_initial_monmap 2026-02-24T18:38:01.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.845+0000 7f1ec1cda640 10 monclient: monmap: 2026-02-24T18:38:01.847 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:01.847 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:01.846564+0000 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:01.846564+0000 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:01.848 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 Processor -- start 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 -- start start 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient: init 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient: _add_conns ranks=[2,0,4,1,3,5] 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1ebc1527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ebc152bb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f1ebc0c6190 0x7f1ebc0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-b con 0x7f1ebc0c6190 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ebc06e300 con 0x7f1ebc152bb0 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ebc06e5e0 con 0x7f1ebc1527d0 2026-02-24T18:38:01.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1ebc06f1f0 con 0x7f1ebc0c6190 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.846+0000 7f1ec1cda640 10 monclient(hunting): authenticate will time out at 1625.527810s 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebaffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1ebc152bb0 0x7f1ebc0c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:56850/0 (socket says 10.20.193.18:56850) 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebaffd640 1 -- 10.20.193.18:0/3727626351 learned_addr learned my addr 10.20.193.18:0/3727626351 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 1 --2- 10.20.193.18:0/3727626351 >> v2:10.20.193.32:3300/0 conn(0x7f1ebc0c6190 0x7f1ebc0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): get_auth_request con 0x7f1ebc0c6190 auth_method 0 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.847+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259339197 0 0) 0x7f1ebc06e300 con 0x7f1ebc152bb0 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): my global_id is 4881 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea00058b0 con 0x7f1ebc152bb0 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914868599 0 0) 0x7f1ebc06e5e0 con 0x7f1ebc1527d0 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 10 monclient(hunting): my global_id is 4847 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.848+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ebc06e300 con 0x7f1ebc1527d0 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_done global_id 4870 payload 274 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 1 -- 10.20.193.18:0/3727626351 >> v1:10.20.193.110:6789/0 conn(0x7f1ebc1527d0 legacy=0x7f1ebc06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 1 -- 10.20.193.18:0/3727626351 >> v1:10.20.193.18:6789/0 conn(0x7f1ebc152bb0 legacy=0x7f1ebc0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 1 monclient: found mon.noname-b 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.849+0000 7f1ebbfff640 1 -- 10.20.193.18:0/3727626351 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc06e8c0 con 0x7f1ebc0c6190 2026-02-24T18:38:01.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.850+0000 7f1ebbfff640 1 --2- 10.20.193.18:0/3727626351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0c8e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06f370 tx=0x7f1eb002f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de220ccda26891e4 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb003c070 con 0x7f1ebc0c6190 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: dump: 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:01.862 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: _finish_auth 0 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 5 monclient: authenticate success, global_id 4870 2026-02-24T18:38:01.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb0037440 con 0x7f1ebc0c6190 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 1 -- 10.20.193.18:0/3727626351 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0037810 con 0x7f1ebc0c6190 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 4 set_mon_vals no callback set 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1eba7fc640 10 monclient: dump: 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.864 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 10 monclient: shutdown 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 -- 10.20.193.18:0/3727626351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc0c6190 msgr2=0x7f1ebc0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/3727626351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0c8e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06f370 tx=0x7f1eb002f230 comp rx=0 tx=0).stop 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 -- 10.20.193.18:0/3727626351 shutdown_connections 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/3727626351 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0c8e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:01.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 -- 10.20.193.18:0/3727626351 >> 10.20.193.18:0/3727626351 conn(0x7f1ebc0cd780 msgr2=0x7f1ebc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 -- 10.20.193.18:0/3727626351 shutdown_connections 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.852+0000 7f1ec1cda640 1 -- 10.20.193.18:0/3727626351 wait complete. 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient: build_initial_monmap 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient: monmap: 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:01.866 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 librados: starting msgr at 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 librados: starting objecter 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 Processor -- start 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 -- start start 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 librados: setting wanted keys 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 librados: calling monclient init 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient: init 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0bb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-c con 0x7f1ebc0c6190 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:01.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 0x7f1ebc1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-b con 0x7f1ebc1527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 0x7f1ebc1ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): picked mon.noname-a con 0x7f1ebc152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc0c93a0 con 0x7f1ebc152bb0 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc06ee80 con 0x7f1ebc1527d0 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc0c9520 con 0x7f1ebc0c6190 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.853+0000 7f1ec1cda640 10 monclient(hunting): authenticate will time out at 1625.534875s 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 0x7f1ebc1ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:01.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 0x7f1ebc1f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 0x7f1ebc1ef5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38558/0 (socket says 10.20.193.18:38558) 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 1 -- 10.20.193.18:0/2988387806 learned_addr learned my addr 10.20.193.18:0/2988387806 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): get_auth_request con 0x7f1ebc152bb0 auth_method 0 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): get_auth_request con 0x7f1ebc1527d0 auth_method 0 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): get_auth_request con 0x7f1ebc0c6190 auth_method 0 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:01.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.854+0000 7f1ebb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 10 monclient(hunting): handle_auth_done global_id 4873 payload 824 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 -- 10.20.193.18:0/2988387806 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1ebc0c6190 msgr2=0x7f1ebc0bb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0bb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 -- 10.20.193.18:0/2988387806 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 msgr2=0x7f1ebc1ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 0x7f1ebc1ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 10 monclient: _finish_hunting 0 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 monclient: found mon.noname-b 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.856+0000 7f1ebaffd640 1 -- 10.20.193.18:0/2988387806 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc1f0010 con 0x7f1ebc1527d0 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.857+0000 7f1ebaffd640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 0x7f1ebc1f09d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c450 tx=0x7f1eac00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.857+0000 7f1e97fff640 10 client.?.objecter ms_handle_connect 0x7f1ebc1527d0 2026-02-24T18:38:01.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.857+0000 7f1e97fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac017070 con 0x7f1ebc1527d0 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: dump: 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:01.872 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: _finish_auth 0 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: _check_auth_tickets 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 5 monclient: authenticate success, global_id 4873 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _renew_subs 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ebc1f02f0 con 0x7f1ebc1527d0 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 client.4873.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _renew_subs 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ebc0ba830 con 0x7f1ebc1527d0 2026-02-24T18:38:01.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 1 librados: init done 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ebc0c5a80 con 0x7f1ebc1527d0 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eac005de0 con 0x7f1ebc1527d0 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac007290 con 0x7f1ebc1527d0 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1e97fff640 10 monclient: dump: 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:01.874 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:01.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.858+0000 7f1eb9ffb640 4 set_mon_vals no callback set 2026-02-24T18:38:01.965 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.964+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1eac007430 con 0x7f1ebc1527d0 2026-02-24T18:38:01.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.964+0000 7f1e97fff640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 0x7f1e8c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:01.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.964+0000 7f1ebb7fe640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 0x7f1e8c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:01.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.964+0000 7f1ebb7fe640 10 monclient: get_auth_request con 0x7f1e8c078e00 auth_method 0 2026-02-24T18:38:01.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.965+0000 7f1ebb7fe640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 0x7f1e8c07b2c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8002410 tx=0x7f1ea8003d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:01.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.965+0000 7f1e97fff640 10 client.4873.objecter ms_handle_connect 0x7f1e8c078e00 2026-02-24T18:38:01.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.966+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f1eac007750 con 0x7f1ebc1527d0 2026-02-24T18:38:01.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.966+0000 7f1e97fff640 10 client.4873.objecter ms_dispatch2 0x7f1ebc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:01.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.966+0000 7f1e97fff640 3 client.4873.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:01.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.966+0000 7f1e97fff640 3 client.4873.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:01.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.966+0000 7f1e97fff640 20 client.4873.objecter dump_active .. 0 homeless 2026-02-24T18:38:01.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.972+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1eac022080 con 0x7f1ebc1527d0 2026-02-24T18:38:01.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.972+0000 7f1e97fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:01.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:01.972+0000 7f1e97fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:02.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.072+0000 7f1ec1cda640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:02.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.072+0000 7f1ec1cda640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:02.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.072+0000 7f1ec1cda640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:02.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.072+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1ebc1492b0 con 0x7f1ebc1527d0 2026-02-24T18:38:02.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.076+0000 7f1e97fff640 1 -- 10.20.193.18:0/2988387806 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1eac0613c0 con 0x7f1ebc1527d0 2026-02-24T18:38:02.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.076+0000 7f1e97fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:02.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.076+0000 7f1e97fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:02.078 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:02.078 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 10 librados: watch_flush enter 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 10 librados: watch_flush exit 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 20 client.4873.objecter shutdown clearing up homeless session... 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 10 client.4873.objecter successfully canceled tick 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 msgr2=0x7f1e8c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:02.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 0x7f1e8c07b2c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8002410 tx=0x7f1ea8003d50 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 10 monclient: shutdown 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 msgr2=0x7f1ebc1f09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 0x7f1ebc1f09d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c450 tx=0x7f1eac00c920 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 shutdown_connections 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1e8c078e00 0x7f1e8c07b2c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1ebc1527d0 0x7f1ebc1f09d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1ebc152bb0 0x7f1ebc1ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 --2- 10.20.193.18:0/2988387806 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1ebc0c6190 0x7f1ebc0bb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 >> 10.20.193.18:0/2988387806 conn(0x7f1ebc0cd780 msgr2=0x7f1ebc1487f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 shutdown_connections 2026-02-24T18:38:02.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 -- 10.20.193.18:0/2988387806 wait complete. 2026-02-24T18:38:02.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:02.077+0000 7f1ec1cda640 1 librados: shutdown 2026-02-24T18:38:05.089 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 10 monclient: get_monmap_and_config 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 10 monclient: build_initial_monmap 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 10 monclient: monmap: 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:05.160268+0000 2026-02-24T18:38:05.161 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:05.160268+0000 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:05.162 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 1 Processor -- start 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 1 -- start start 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.159+0000 7efdacdcb640 10 monclient: init 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient: _add_conns ranks=[1,5,4,2,0,3] 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efda806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7efda80c99a0 0x7efda8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-c con 0x7efda80c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efda815a5e0 0x7efda814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-b con 0x7efda815a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda80c7e90 con 0x7efda806ff20 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7efda8159a40 con 0x7efda815a5e0 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:05.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7efda80c8170 con 0x7efda80c99a0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.160+0000 7efdacdcb640 10 monclient(hunting): authenticate will time out at 1628.841629s 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6575640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7efda806ff20 0x7efda8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51990/0 (socket says 10.20.193.18:51990) 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6575640 1 -- 10.20.193.18:0/1799965922 learned_addr learned my addr 10.20.193.18:0/1799965922 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 1 --2- 10.20.193.18:0/1799965922 >> v2:10.20.193.110:3300/0 conn(0x7efda80c99a0 0x7efda8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): get_auth_request con 0x7efda80c99a0 auth_method 0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 1 --2- 10.20.193.18:0/1799965922 >> v2:10.20.193.32:3300/0 conn(0x7efda815a5e0 0x7efda814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): get_auth_request con 0x7efda815a5e0 auth_method 0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.161+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 10 monclient(hunting): handle_auth_done global_id 4853 payload 274 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 1 -- 10.20.193.18:0/1799965922 >> v2:10.20.193.32:3300/0 conn(0x7efda815a5e0 msgr2=0x7efda814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 1 --2- 10.20.193.18:0/1799965922 >> v2:10.20.193.32:3300/0 conn(0x7efda815a5e0 0x7efda814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 1 -- 10.20.193.18:0/1799965922 >> v1:10.20.193.32:6789/0 conn(0x7efda806ff20 legacy=0x7efda8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 10 monclient: _finish_hunting 0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 1 monclient: found mon.noname-c 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.162+0000 7efda5d74640 1 -- 10.20.193.18:0/1799965922 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda80c8450 con 0x7efda80c99a0 2026-02-24T18:38:05.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.163+0000 7efda5d74640 1 --2- 10.20.193.18:0/1799965922 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda8070300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd900098a0 tx=0x7efd9002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b076fae5e8a5c2 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 1 -- 10.20.193.18:0/1799965922 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9003c070 con 0x7efda80c99a0 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: dump: 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:05.166 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: _finish_auth 0 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: _check_auth_tickets 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 1 -- 10.20.193.18:0/1799965922 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9002fa40 con 0x7efda80c99a0 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 5 monclient: authenticate success, global_id 4853 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 1 -- 10.20.193.18:0/1799965922 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9002fe10 con 0x7efda80c99a0 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 4 set_mon_vals no callback set 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efda5573640 10 monclient: dump: 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:05.167 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 10 monclient: shutdown 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 -- 10.20.193.18:0/1799965922 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 msgr2=0x7efda8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 --2- 10.20.193.18:0/1799965922 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda8070300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd900098a0 tx=0x7efd9002ee10 comp rx=0 tx=0).stop 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 -- 10.20.193.18:0/1799965922 shutdown_connections 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 --2- 10.20.193.18:0/1799965922 >> v2:10.20.193.32:3300/0 conn(0x7efda815a5e0 0x7efda814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 --2- 10.20.193.18:0/1799965922 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda8070300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 -- 10.20.193.18:0/1799965922 >> 10.20.193.18:0/1799965922 conn(0x7efda80cd780 msgr2=0x7efda80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 -- 10.20.193.18:0/1799965922 shutdown_connections 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.165+0000 7efdacdcb640 1 -- 10.20.193.18:0/1799965922 wait complete. 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient: build_initial_monmap 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient: monmap: 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 librados: starting msgr at 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 librados: starting objecter 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 Processor -- start 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 -- start start 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 librados: setting wanted keys 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 librados: calling monclient init 2026-02-24T18:38:05.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient: init 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efda806ff20 0x7efda81f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-a con 0x7efda806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-c con 0x7efda80c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efda815a5e0 0x7efda81ef910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): picked mon.noname-b con 0x7efda815a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efda8150350 con 0x7efda806ff20 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efda80c8a10 con 0x7efda815a5e0 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efda81504d0 con 0x7efda80c99a0 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.166+0000 7efdacdcb640 10 monclient(hunting): authenticate will time out at 1628.847988s 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efda806ff20 0x7efda81f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efda815a5e0 0x7efda81ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:46744/0 (socket says 10.20.193.18:46744) 2026-02-24T18:38:05.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 1 -- 10.20.193.18:0/4126775106 learned_addr learned my addr 10.20.193.18:0/4126775106 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): get_auth_request con 0x7efda80c99a0 auth_method 0 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): get_auth_request con 0x7efda806ff20 auth_method 0 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): get_auth_request con 0x7efda815a5e0 auth_method 0 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.167+0000 7efda6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 10 monclient(hunting): handle_auth_done global_id 4856 payload 824 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 -- 10.20.193.18:0/4126775106 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efda815a5e0 msgr2=0x7efda81ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efda815a5e0 0x7efda81ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 -- 10.20.193.18:0/4126775106 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efda806ff20 msgr2=0x7efda81f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efda806ff20 0x7efda81f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 10 monclient: _finish_hunting 0 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 monclient: found mon.noname-c 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.169+0000 7efda5d74640 1 -- 10.20.193.18:0/4126775106 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda81f24c0 con 0x7efda80c99a0 2026-02-24T18:38:05.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.170+0000 7efda5d74640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd900099d0 tx=0x7efd900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:05.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.170+0000 7efd8effd640 10 client.?.objecter ms_handle_connect 0x7efda80c99a0 2026-02-24T18:38:05.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.170+0000 7efd8effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9003c040 con 0x7efda80c99a0 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: dump: 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: _finish_auth 0 2026-02-24T18:38:05.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 5 monclient: authenticate success, global_id 4856 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 monclient: _renew_subs 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda814c980 con 0x7efda80c99a0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 client.4856.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 monclient: _renew_subs 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda80c2170 con 0x7efda80c99a0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 1 librados: init done 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90047070 con 0x7efda80c99a0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90042650 con 0x7efda80c99a0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efd8effd640 10 monclient: dump: 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efda4d72640 4 set_mon_vals no callback set 2026-02-24T18:38:05.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.171+0000 7efdacdcb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:05.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.172+0000 7efdacdcb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:05.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.172+0000 7efdacdcb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.172+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda8150860 con 0x7efda80c99a0 2026-02-24T18:38:05.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.183+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efd900427f0 con 0x7efda80c99a0 2026-02-24T18:38:05.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.183+0000 7efd8effd640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 0x7efd7407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.183+0000 7efda6575640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 0x7efd7407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.183+0000 7efda6575640 10 monclient: get_auth_request con 0x7efd74078e00 auth_method 0 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efda6575640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 0x7efd7407b2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007d90 tx=0x7efd9c007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 10 client.4856.objecter ms_handle_connect 0x7efd74078e00 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7efd900bd180 con 0x7efda80c99a0 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 10 client.4856.objecter ms_dispatch2 0x7efda8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 3 client.4856.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 3 client.4856.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:05.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.184+0000 7efd8effd640 20 client.4856.objecter dump_active .. 0 homeless 2026-02-24T18:38:05.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.286+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efd900879e0 con 0x7efda80c99a0 2026-02-24T18:38:05.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.286+0000 7efd8effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:05.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.286+0000 7efd8effd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:05.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.387+0000 7efdacdcb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:05.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.387+0000 7efdacdcb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:05.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.387+0000 7efdacdcb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:05.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.387+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efda814f3b0 con 0x7efda80c99a0 2026-02-24T18:38:05.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efd8effd640 1 -- 10.20.193.18:0/4126775106 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7efd9008bce0 con 0x7efda80c99a0 2026-02-24T18:38:05.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efd8effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:05.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efd8effd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:05.391 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:05.391 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:05.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 10 librados: watch_flush enter 2026-02-24T18:38:05.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 10 librados: watch_flush exit 2026-02-24T18:38:05.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 20 client.4856.objecter shutdown clearing up homeless session... 2026-02-24T18:38:05.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 10 client.4856.objecter successfully canceled tick 2026-02-24T18:38:05.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 msgr2=0x7efd7407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 0x7efd7407b2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007d90 tx=0x7efd9c007ce0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 10 monclient: shutdown 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 msgr2=0x7efda814c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd900099d0 tx=0x7efd900047c0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 shutdown_connections 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd74078e00 0x7efd7407b2c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efda815a5e0 0x7efda81ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efda80c99a0 0x7efda814c440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 --2- 10.20.193.18:0/4126775106 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efda806ff20 0x7efda81f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:05.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 >> 10.20.193.18:0/4126775106 conn(0x7efda80cd780 msgr2=0x7efda8157840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:05.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.390+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 shutdown_connections 2026-02-24T18:38:05.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.391+0000 7efdacdcb640 1 -- 10.20.193.18:0/4126775106 wait complete. 2026-02-24T18:38:05.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:05.391+0000 7efdacdcb640 1 librados: shutdown 2026-02-24T18:38:08.403 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.472+0000 7f6d969fe640 10 monclient: get_monmap_and_config 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.472+0000 7f6d969fe640 10 monclient: build_initial_monmap 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.472+0000 7f6d969fe640 10 monclient: monmap: 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:08.473403+0000 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:08.473403+0000 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:08.474 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:08.475 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:08.475 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:08.475 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:08.475 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:08.475 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.472+0000 7f6d969fe640 1 Processor -- start 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 1 -- start start 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient: init 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient: _add_conns ranks=[0,2,4,5,3,1] 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d9006ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6d900c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f6d9015a5e0 0x7f6d90070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-b con 0x7f6d9015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900c7e90 con 0x7f6d9006ff20 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900c8170 con 0x7f6d900c99a0 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6d90159a40 con 0x7f6d9015a5e0 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d969fe640 10 monclient(hunting): authenticate will time out at 1632.154561s 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d8ffff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f6d9006ff20 0x7f6d90159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:56860/0 (socket says 10.20.193.18:56860) 2026-02-24T18:38:08.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.473+0000 7f6d8ffff640 1 -- 10.20.193.18:0/2518715089 learned_addr learned my addr 10.20.193.18:0/2518715089 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 1 --2- 10.20.193.18:0/2518715089 >> v2:10.20.193.32:3300/0 conn(0x7f6d9015a5e0 0x7f6d90070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): get_auth_request con 0x7f6d9015a5e0 auth_method 0 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.474+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042721890 0 0) 0x7f6d900c7e90 con 0x7f6d9006ff20 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _negotiate 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): my global_id is 4890 2026-02-24T18:38:08.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d74003620 con 0x7f6d9006ff20 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187579214 0 0) 0x7f6d900c8170 con 0x7f6d900c99a0 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _negotiate 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 10 monclient(hunting): my global_id is 4859 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d900c7e90 con 0x7f6d900c99a0 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 10 monclient(hunting): handle_auth_done global_id 4882 payload 274 2026-02-24T18:38:08.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 1 -- 10.20.193.18:0/2518715089 >> v1:10.20.193.110:6789/0 conn(0x7f6d900c99a0 legacy=0x7f6d9014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 1 -- 10.20.193.18:0/2518715089 >> v1:10.20.193.18:6789/0 conn(0x7f6d9006ff20 legacy=0x7f6d90159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 10 monclient: _finish_hunting 0 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 1 monclient: found mon.noname-b 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.475+0000 7f6d94f74640 1 -- 10.20.193.18:0/2518715089 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d900c8450 con 0x7f6d9015a5e0 2026-02-24T18:38:08.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.477+0000 7f6d94f74640 1 --2- 10.20.193.18:0/2518715089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9015a5e0 0x7f6d90070300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009dc0 tx=0x7f6d8402f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd6fb645ac93f5dd server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8403c070 con 0x7f6d9015a5e0 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: dump: 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: _finish_auth 0 2026-02-24T18:38:08.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d969fe640 5 monclient: authenticate success, global_id 4882 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d969fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d84037440 con 0x7f6d9015a5e0 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 1 -- 10.20.193.18:0/2518715089 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840377f0 con 0x7f6d9015a5e0 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d8effd640 10 monclient: dump: 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:08.484 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d969fe640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.478+0000 7f6d969fe640 4 set_mon_vals no callback set 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 10 monclient: shutdown 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 -- 10.20.193.18:0/2518715089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9015a5e0 msgr2=0x7f6d90070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 --2- 10.20.193.18:0/2518715089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9015a5e0 0x7f6d90070300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009dc0 tx=0x7f6d8402f280 comp rx=0 tx=0).stop 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 -- 10.20.193.18:0/2518715089 shutdown_connections 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 --2- 10.20.193.18:0/2518715089 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9015a5e0 0x7f6d90070300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 -- 10.20.193.18:0/2518715089 >> 10.20.193.18:0/2518715089 conn(0x7f6d900cd780 msgr2=0x7f6d900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 -- 10.20.193.18:0/2518715089 shutdown_connections 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 -- 10.20.193.18:0/2518715089 wait complete. 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 10 monclient: build_initial_monmap 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 10 monclient: monmap: 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.485 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 librados: starting msgr at 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 librados: starting objecter 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.479+0000 7f6d969fe640 1 Processor -- start 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 -- start start 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 librados: setting wanted keys 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 librados: calling monclient init 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient: init 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9006ff20 0x7f6d90154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-b con 0x7f6d9006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6d900c99a0 0x7f6d9014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-c con 0x7f6d900c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): picked mon.noname-a con 0x7f6d9015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90150520 con 0x7f6d9015a5e0 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:08.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900c8a10 con 0x7f6d9006ff20 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901506a0 con 0x7f6d900c99a0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d969fe640 10 monclient(hunting): authenticate will time out at 1632.161475s 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d8f7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6d900c99a0 0x7f6d9014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d8ffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9006ff20 0x7f6d90154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:38580/0 (socket says 10.20.193.18:38580) 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 1 -- 10.20.193.18:0/1638234271 learned_addr learned my addr 10.20.193.18:0/1638234271 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 10 monclient(hunting): get_auth_request con 0x7f6d9015a5e0 auth_method 0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.480+0000 7f6d94f74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): get_auth_request con 0x7f6d900c99a0 auth_method 0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): get_auth_request con 0x7f6d9006ff20 auth_method 0 2026-02-24T18:38:08.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d94f74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:08.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.481+0000 7f6d8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 10 monclient(hunting): handle_auth_done global_id 4893 payload 824 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 -- 10.20.193.18:0/1638234271 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6d900c99a0 msgr2=0x7f6d9014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6d900c99a0 0x7f6d9014bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 -- 10.20.193.18:0/1638234271 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9006ff20 msgr2=0x7f6d90154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9006ff20 0x7f6d90154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 10 monclient: _finish_hunting 0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 monclient: found mon.noname-a 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.482+0000 7f6d94f74640 1 -- 10.20.193.18:0/1638234271 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d90155560 con 0x7f6d9015a5e0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.483+0000 7f6d94f74640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009ef0 tx=0x7f6d84004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.483+0000 7f6d8cff9640 10 client.?.objecter ms_handle_connect 0x7f6d9015a5e0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.483+0000 7f6d8cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8403c050 con 0x7f6d9015a5e0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: dump: 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: _finish_auth 0 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:08.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 5 monclient: authenticate success, global_id 4893 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d84048070 con 0x7f6d9015a5e0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84009450 con 0x7f6d9015a5e0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8cff9640 10 monclient: dump: 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d8e7fc640 4 set_mon_vals no callback set 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 10 monclient: _renew_subs 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d90155840 con 0x7f6d9015a5e0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 10 client.4893.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 10 monclient: _renew_subs 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d90155e00 con 0x7f6d9015a5e0 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.485+0000 7f6d969fe640 1 librados: init done 2026-02-24T18:38:08.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.486+0000 7f6d969fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:08.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.486+0000 7f6d969fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:08.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.486+0000 7f6d969fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:08.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.486+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d90070300 con 0x7f6d9015a5e0 2026-02-24T18:38:08.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.593+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6d84004040 con 0x7f6d9015a5e0 2026-02-24T18:38:08.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.593+0000 7f6d8cff9640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 0x7f6d6007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:08.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.593+0000 7f6d8ffff640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 0x7f6d6007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:08.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.593+0000 7f6d8ffff640 10 monclient: get_auth_request con 0x7f6d60078e00 auth_method 0 2026-02-24T18:38:08.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.594+0000 7f6d8ffff640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 0x7f6d6007b2c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c007b60 tx=0x7f6d7c007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:08.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.594+0000 7f6d8cff9640 10 client.4893.objecter ms_handle_connect 0x7f6d60078e00 2026-02-24T18:38:08.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.603+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f6d84083fe0 con 0x7f6d9015a5e0 2026-02-24T18:38:08.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.603+0000 7f6d8cff9640 10 client.4893.objecter ms_dispatch2 0x7f6d90000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.603+0000 7f6d8cff9640 3 client.4893.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.603+0000 7f6d8cff9640 3 client.4893.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.603+0000 7f6d8cff9640 20 client.4893.objecter dump_active .. 0 homeless 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.604+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6d84087fa0 con 0x7f6d9015a5e0 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.604+0000 7f6d8cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:08.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.604+0000 7f6d8cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:08.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.704+0000 7f6d969fe640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:08.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.704+0000 7f6d969fe640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:08.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.704+0000 7f6d969fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:08.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.704+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6d90158740 con 0x7f6d9015a5e0 2026-02-24T18:38:08.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.706+0000 7f6d8cff9640 1 -- 10.20.193.18:0/1638234271 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6d90158740 con 0x7f6d9015a5e0 2026-02-24T18:38:08.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.706+0000 7f6d8cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:08.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.706+0000 7f6d8cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:08.709 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:08.709 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 10 librados: watch_flush enter 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 10 librados: watch_flush exit 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 20 client.4893.objecter shutdown clearing up homeless session... 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 10 client.4893.objecter successfully canceled tick 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 msgr2=0x7f6d6007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 0x7f6d6007b2c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c007b60 tx=0x7f6d7c007ab0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 10 monclient: shutdown 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 msgr2=0x7f6d900c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:08.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009ef0 tx=0x7f6d84004b40 comp rx=0 tx=0).stop 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 shutdown_connections 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6d60078e00 0x7f6d6007b2c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6d9015a5e0 0x7f6d900c25f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6d900c99a0 0x7f6d9014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 --2- 10.20.193.18:0/1638234271 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6d9006ff20 0x7f6d90154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 >> 10.20.193.18:0/1638234271 conn(0x7f6d900cd780 msgr2=0x7f6d90158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 shutdown_connections 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 -- 10.20.193.18:0/1638234271 wait complete. 2026-02-24T18:38:08.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:08.707+0000 7f6d969fe640 1 librados: shutdown 2026-02-24T18:38:11.719 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:11.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 10 monclient: get_monmap_and_config 2026-02-24T18:38:11.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 10 monclient: build_initial_monmap 2026-02-24T18:38:11.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 10 monclient: monmap: 2026-02-24T18:38:11.788 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:11.787151+0000 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:11.787151+0000 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:11.789 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:11.790 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 1 Processor -- start 2026-02-24T18:38:11.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 1 -- start start 2026-02-24T18:38:11.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.786+0000 7f60b593e640 10 monclient: init 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient: _add_conns ranks=[3,4,5,2,1,0] 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f60b006ff20 0x7f60b0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-a con 0x7f60b006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f60b00c99a0 0x7f60b0159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-b con 0x7f60b00c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f60b015a410 0x7f60b014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-c con 0x7f60b015a410 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f60b0159a40 con 0x7f60b006ff20 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f60b015a860 con 0x7f60b00c99a0 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.793 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f60b00c7e90 con 0x7f60b015a410 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60b593e640 10 monclient(hunting): authenticate will time out at 1635.468262s 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f60b006ff20 0x7f60b0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f60b006ff20 0x7f60b0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47040/0 (socket says 10.20.193.18:47040) 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 1 -- 10.20.193.18:0/1845094538 learned_addr learned my addr 10.20.193.18:0/1845094538 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): get_auth_request con 0x7f60b006ff20 auth_method 0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60ae7fc640 1 --2- 10.20.193.18:0/1845094538 >> v2:10.20.193.32:3300/0 conn(0x7f60b00c99a0 0x7f60b0159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60ae7fc640 10 monclient(hunting): get_auth_request con 0x7f60b00c99a0 auth_method 0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.787+0000 7f60ae7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60ae7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 10 monclient(hunting): handle_auth_done global_id 4896 payload 274 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 -- 10.20.193.18:0/1845094538 >> v2:10.20.193.110:3300/0 conn(0x7f60b015a410 msgr2=0x7f60b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 --2- 10.20.193.18:0/1845094538 >> v2:10.20.193.110:3300/0 conn(0x7f60b015a410 0x7f60b014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 -- 10.20.193.18:0/1845094538 >> v2:10.20.193.32:3300/0 conn(0x7f60b00c99a0 msgr2=0x7f60b0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 --2- 10.20.193.18:0/1845094538 >> v2:10.20.193.32:3300/0 conn(0x7f60b00c99a0 0x7f60b0159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 10 monclient: _finish_hunting 0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 monclient: found mon.noname-a 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.788+0000 7f60aeffd640 1 -- 10.20.193.18:0/1845094538 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b00c8170 con 0x7f60b006ff20 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.789+0000 7f60aeffd640 1 --2- 10.20.193.18:0/1845094538 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b006ff20 0x7f60b0070300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60b00c5380 tx=0x7f60a402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d9125cb32b814 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 1 -- 10.20.193.18:0/1845094538 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a403c070 con 0x7f60b006ff20 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: dump: 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.794 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: _finish_auth 0 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: _check_auth_tickets 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 1 -- 10.20.193.18:0/1845094538 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a402fb50 con 0x7f60b006ff20 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 5 monclient: authenticate success, global_id 4896 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 1 -- 10.20.193.18:0/1845094538 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4037440 con 0x7f60b006ff20 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60adffb640 10 monclient: dump: 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 4 set_mon_vals no callback set 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 10 monclient: shutdown 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 -- 10.20.193.18:0/1845094538 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b006ff20 msgr2=0x7f60b0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 --2- 10.20.193.18:0/1845094538 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b006ff20 0x7f60b0070300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60b00c5380 tx=0x7f60a402eed0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 -- 10.20.193.18:0/1845094538 shutdown_connections 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 --2- 10.20.193.18:0/1845094538 >> v2:10.20.193.110:3300/0 conn(0x7f60b015a410 0x7f60b014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 --2- 10.20.193.18:0/1845094538 >> v2:10.20.193.32:3300/0 conn(0x7f60b00c99a0 0x7f60b0159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 --2- 10.20.193.18:0/1845094538 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b006ff20 0x7f60b0070300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.791+0000 7f60b593e640 1 -- 10.20.193.18:0/1845094538 >> 10.20.193.18:0/1845094538 conn(0x7f60b00cd780 msgr2=0x7f60b00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 1 -- 10.20.193.18:0/1845094538 shutdown_connections 2026-02-24T18:38:11.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 1 -- 10.20.193.18:0/1845094538 wait complete. 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 10 monclient: build_initial_monmap 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 10 monclient: monmap: 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:11.796 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 1 librados: starting msgr at 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.792+0000 7f60b593e640 1 librados: starting objecter 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 Processor -- start 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 -- start start 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 librados: setting wanted keys 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 librados: calling monclient init 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient: init 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f60b006ff20 0x7f60b01f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-c con 0x7f60b006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f60b00c99a0 0x7f60b006b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-b con 0x7f60b00c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): picked mon.noname-a con 0x7f60b015a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00c8a10 con 0x7f60b015a410 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00c8730 con 0x7f60b00c99a0 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0150350 con 0x7f60b006ff20 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60b593e640 10 monclient(hunting): authenticate will time out at 1635.474611s 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60ae7fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f60b00c99a0 0x7f60b006b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47044/0 (socket says 10.20.193.18:47044) 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 1 -- 10.20.193.18:0/3289313905 learned_addr learned my addr 10.20.193.18:0/3289313905 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 10 monclient(hunting): get_auth_request con 0x7f60b015a410 auth_method 0 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.793+0000 7f60af7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): get_auth_request con 0x7f60b00c99a0 auth_method 0 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:11.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.794+0000 7f60ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 10 monclient(hunting): handle_auth_done global_id 4899 payload 824 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 -- 10.20.193.18:0/3289313905 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f60b006ff20 msgr2=0x7f60b01f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f60b006ff20 0x7f60b01f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 -- 10.20.193.18:0/3289313905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f60b00c99a0 msgr2=0x7f60b006b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f60b00c99a0 0x7f60b006b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 monclient: found mon.noname-a 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.795+0000 7f60af7fe640 1 -- 10.20.193.18:0/3289313905 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01f77a0 con 0x7f60b015a410 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.796+0000 7f60af7fe640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b970 tx=0x7f60a000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.796+0000 7f6087fff640 10 client.?.objecter ms_handle_connect 0x7f60b015a410 2026-02-24T18:38:11.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.796+0000 7f6087fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000c790 con 0x7f60b015a410 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: dump: 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.808 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: _finish_auth 0 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: _check_auth_tickets 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a0010070 con 0x7f60b015a410 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:11.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000cb90 con 0x7f60b015a410 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 5 monclient: authenticate success, global_id 4899 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _renew_subs 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60ad7fa640 4 set_mon_vals no callback set 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01f7a80 con 0x7f60b015a410 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f6087fff640 10 monclient: dump: 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:11.810 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 client.4899.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _renew_subs 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01f5120 con 0x7f60b015a410 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 1 librados: init done 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:11.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:11.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:11.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.805+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b01506e0 con 0x7f60b015a410 2026-02-24T18:38:11.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.918+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f60a001d030 con 0x7f60b015a410 2026-02-24T18:38:11.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.918+0000 7f6087fff640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 0x7f608007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:11.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.918+0000 7f60aeffd640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 0x7f608007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:11.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.918+0000 7f60aeffd640 10 monclient: get_auth_request con 0x7f6080078e00 auth_method 0 2026-02-24T18:38:11.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.919+0000 7f6087fff640 10 client.4899.objecter ms_handle_connect 0x7f6080078e00 2026-02-24T18:38:11.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.919+0000 7f60aeffd640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 0x7f608007b2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f60b00709c0 tx=0x7f60a4031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f60a000cd30 con 0x7f60b015a410 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 10 client.4899.objecter ms_dispatch2 0x7f60b0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 3 client.4899.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 3 client.4899.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 20 client.4899.objecter dump_active .. 0 homeless 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60a0023020 con 0x7f60b015a410 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:11.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:11.926+0000 7f6087fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:12.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.026+0000 7f60b593e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:12.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.026+0000 7f60b593e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:12.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.026+0000 7f60b593e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:12.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.026+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f60b014ec40 con 0x7f60b015a410 2026-02-24T18:38:12.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.028+0000 7f6087fff640 1 -- 10.20.193.18:0/3289313905 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f60a0062230 con 0x7f60b015a410 2026-02-24T18:38:12.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.028+0000 7f6087fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:12.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.028+0000 7f6087fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:12.030 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:12.031 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:12.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 10 librados: watch_flush enter 2026-02-24T18:38:12.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 10 librados: watch_flush exit 2026-02-24T18:38:12.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 20 client.4899.objecter shutdown clearing up homeless session... 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 10 client.4899.objecter successfully canceled tick 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 msgr2=0x7f608007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 0x7f608007b2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f60b00709c0 tx=0x7f60a4031000 comp rx=0 tx=0).stop 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 10 monclient: shutdown 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 msgr2=0x7f60b01f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b970 tx=0x7f60a000be40 comp rx=0 tx=0).stop 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 shutdown_connections 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6080078e00 0x7f608007b2c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:12.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f60b00c99a0 0x7f60b006b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f60b015a410 0x7f60b01f5920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 --2- 10.20.193.18:0/3289313905 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f60b006ff20 0x7f60b01f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 >> 10.20.193.18:0/3289313905 conn(0x7f60b00cd780 msgr2=0x7f60b014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 shutdown_connections 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 -- 10.20.193.18:0/3289313905 wait complete. 2026-02-24T18:38:12.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:12.030+0000 7f60b593e640 1 librados: shutdown 2026-02-24T18:38:15.042 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:15.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 10 monclient: get_monmap_and_config 2026-02-24T18:38:15.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 10 monclient: build_initial_monmap 2026-02-24T18:38:15.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 10 monclient: monmap: 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:15.111043+0000 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:15.111043+0000 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:15.112 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 1 Processor -- start 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 1 -- start start 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.110+0000 7f969ec2b640 10 monclient: init 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient: _add_conns ranks=[3,5,4,0,1,2] 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f96981527d0 0x7f969806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-a con 0x7f96981527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9698152bb0 0x7f96980c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-c con 0x7f9698152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f96980c5fc0 0x7f96980c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-b con 0x7f96980c5fc0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f969806f1f0 con 0x7f96981527d0 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.114 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f96980c6470 con 0x7f96980c5fc0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f969806e300 con 0x7f9698152bb0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969ec2b640 10 monclient(hunting): authenticate will time out at 1638.792194s 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f96981527d0 0x7f969806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969d1a1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f96980c5fc0 0x7f96980c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f96981527d0 0x7f969806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47050/0 (socket says 10.20.193.18:47050) 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 1 -- 10.20.193.18:0/3315744767 learned_addr learned my addr 10.20.193.18:0/3315744767 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): get_auth_request con 0x7f96981527d0 auth_method 0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f968ffff640 1 --2- 10.20.193.18:0/3315744767 >> v2:10.20.193.110:3300/0 conn(0x7f9698152bb0 0x7f96980c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969d1a1640 10 monclient(hunting): get_auth_request con 0x7f96980c5fc0 auth_method 0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969d1a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969d1a1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969d1a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.111+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): get_auth_request con 0x7f9698152bb0 auth_method 0 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.115 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 10 monclient(hunting): handle_auth_done global_id 4902 payload 274 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 -- 10.20.193.18:0/3315744767 >> v2:10.20.193.110:3300/0 conn(0x7f9698152bb0 msgr2=0x7f96980c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 --2- 10.20.193.18:0/3315744767 >> v2:10.20.193.110:3300/0 conn(0x7f9698152bb0 0x7f96980c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 -- 10.20.193.18:0/3315744767 >> v2:10.20.193.32:3300/0 conn(0x7f96980c5fc0 msgr2=0x7f96980c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 --2- 10.20.193.18:0/3315744767 >> v2:10.20.193.32:3300/0 conn(0x7f96980c5fc0 0x7f96980c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 10 monclient: _finish_hunting 0 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 monclient: found mon.noname-a 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.112+0000 7f969c9a0640 1 -- 10.20.193.18:0/3315744767 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969806e5e0 con 0x7f96981527d0 2026-02-24T18:38:15.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.113+0000 7f969c9a0640 1 --2- 10.20.193.18:0/3315744767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96981527d0 0x7f969806da60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9698158940 tx=0x7f968802ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c29d2c08b26293 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 1 -- 10.20.193.18:0/3315744767 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968803c070 con 0x7f96981527d0 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: dump: 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 1 -- 10.20.193.18:0/3315744767 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968802fb50 con 0x7f96981527d0 2026-02-24T18:38:15.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f969ec2b640 5 monclient: authenticate success, global_id 4902 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f969ec2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 1 -- 10.20.193.18:0/3315744767 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688037440 con 0x7f96981527d0 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f969ec2b640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f969ec2b640 4 set_mon_vals no callback set 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.115+0000 7f968f7fe640 10 monclient: dump: 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 10 monclient: shutdown 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 -- 10.20.193.18:0/3315744767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96981527d0 msgr2=0x7f969806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 --2- 10.20.193.18:0/3315744767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96981527d0 0x7f969806da60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9698158940 tx=0x7f968802ee90 comp rx=0 tx=0).stop 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 -- 10.20.193.18:0/3315744767 shutdown_connections 2026-02-24T18:38:15.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 --2- 10.20.193.18:0/3315744767 >> v2:10.20.193.32:3300/0 conn(0x7f96980c5fc0 0x7f96980c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 --2- 10.20.193.18:0/3315744767 >> v2:10.20.193.110:3300/0 conn(0x7f9698152bb0 0x7f96980c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 --2- 10.20.193.18:0/3315744767 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96981527d0 0x7f969806da60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 -- 10.20.193.18:0/3315744767 >> 10.20.193.18:0/3315744767 conn(0x7f96980cd780 msgr2=0x7f96980cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 -- 10.20.193.18:0/3315744767 shutdown_connections 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 -- 10.20.193.18:0/3315744767 wait complete. 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 10 monclient: build_initial_monmap 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 10 monclient: monmap: 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:15.120 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 librados: starting msgr at 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.116+0000 7f969ec2b640 1 librados: starting objecter 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 Processor -- start 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 -- start start 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 librados: setting wanted keys 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 librados: calling monclient init 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient: init 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:15.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 0x7f96980744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-a con 0x7f96980c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f96981527d0 0x7f96980749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-b con 0x7f96981527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 0x7f9698163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): picked mon.noname-c con 0x7f9698152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f969806ee80 con 0x7f96980c5fc0 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f969806eba0 con 0x7f96981527d0 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:15.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f969806dfa0 con 0x7f9698152bb0 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969ec2b640 10 monclient(hunting): authenticate will time out at 1638.798520s 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 0x7f96980744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 0x7f96980744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47054/0 (socket says 10.20.193.18:47054) 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 1 -- 10.20.193.18:0/4119445972 learned_addr learned my addr 10.20.193.18:0/4119445972 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 10 monclient(hunting): get_auth_request con 0x7f96980c5fc0 auth_method 0 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969c9a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f969d1a1640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 0x7f9698163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.117+0000 7f968ffff640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f96981527d0 0x7f96980749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): get_auth_request con 0x7f96981527d0 auth_method 0 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): get_auth_request con 0x7f9698152bb0 auth_method 0 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969c9a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f969d1a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.118+0000 7f968ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 10 monclient(hunting): handle_auth_done global_id 4868 payload 824 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 -- 10.20.193.18:0/4119445972 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f96981527d0 msgr2=0x7f96980749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f96981527d0 0x7f96980749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 -- 10.20.193.18:0/4119445972 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 msgr2=0x7f96980744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 0x7f96980744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 10 monclient: _finish_hunting 0 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 monclient: found mon.noname-c 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:15.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:15.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.119+0000 7f969d1a1640 1 -- 10.20.193.18:0/4119445972 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9698143b40 con 0x7f9698152bb0 2026-02-24T18:38:15.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.120+0000 7f969d1a1640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 0x7f9698163d30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f969400efc0 tx=0x7f969400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:15.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.120+0000 7f968d7fa640 10 client.?.objecter ms_handle_connect 0x7f9698152bb0 2026-02-24T18:38:15.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.120+0000 7f968d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694018020 con 0x7f9698152bb0 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: dump: 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.130 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9694014480 con 0x7f9698152bb0 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 5 monclient: authenticate success, global_id 4868 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:15.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694014870 con 0x7f9698152bb0 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968d7fa640 10 monclient: dump: 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:15.132 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 10 monclient: _renew_subs 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f968effd640 4 set_mon_vals no callback set 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9698142fc0 con 0x7f9698152bb0 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 10 client.4868.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 10 monclient: _renew_subs 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9698143580 con 0x7f9698152bb0 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.122+0000 7f969ec2b640 1 librados: init done 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.123+0000 7f969ec2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.123+0000 7f969ec2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:15.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.123+0000 7f969ec2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:15.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.123+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969806da60 con 0x7f9698152bb0 2026-02-24T18:38:15.135 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.134+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9694014a10 con 0x7f9698152bb0 2026-02-24T18:38:15.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.134+0000 7f968d7fa640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 0x7f966807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:15.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.134+0000 7f969c9a0640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 0x7f966807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:15.136 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.134+0000 7f969c9a0640 10 monclient: get_auth_request con 0x7f9668078e00 auth_method 0 2026-02-24T18:38:15.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.135+0000 7f969c9a0640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 0x7f966807b2c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9688009730 tx=0x7f9688039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:15.137 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.135+0000 7f968d7fa640 10 client.4868.objecter ms_handle_connect 0x7f9668078e00 2026-02-24T18:38:15.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.136+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f969405ef90 con 0x7f9698152bb0 2026-02-24T18:38:15.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.136+0000 7f968d7fa640 10 client.4868.objecter ms_dispatch2 0x7f9698000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:15.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.136+0000 7f968d7fa640 3 client.4868.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:15.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.136+0000 7f968d7fa640 3 client.4868.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:15.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.136+0000 7f968d7fa640 20 client.4868.objecter dump_active .. 0 homeless 2026-02-24T18:38:15.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.237+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9694022030 con 0x7f9698152bb0 2026-02-24T18:38:15.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.237+0000 7f968d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:15.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.237+0000 7f968d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:15.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.341+0000 7f969ec2b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:15.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.341+0000 7f969ec2b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:15.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.341+0000 7f969ec2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:15.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.341+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9698070650 con 0x7f9698152bb0 2026-02-24T18:38:15.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.344+0000 7f968d7fa640 1 -- 10.20.193.18:0/4119445972 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9694010040 con 0x7f9698152bb0 2026-02-24T18:38:15.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.344+0000 7f968d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:15.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.344+0000 7f968d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:15.346 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:15.346 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:15.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 10 librados: watch_flush enter 2026-02-24T18:38:15.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 10 librados: watch_flush exit 2026-02-24T18:38:15.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 20 client.4868.objecter shutdown clearing up homeless session... 2026-02-24T18:38:15.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 10 client.4868.objecter successfully canceled tick 2026-02-24T18:38:15.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 msgr2=0x7f966807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 0x7f966807b2c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9688009730 tx=0x7f9688039040 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 10 monclient: shutdown 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 msgr2=0x7f9698163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 0x7f9698163d30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f969400efc0 tx=0x7f969400c490 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 shutdown_connections 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9668078e00 0x7f966807b2c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f96981527d0 0x7f96980749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9698152bb0 0x7f9698163d30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 --2- 10.20.193.18:0/4119445972 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f96980c5fc0 0x7f96980744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 >> 10.20.193.18:0/4119445972 conn(0x7f96980cd780 msgr2=0x7f969814fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:15.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 shutdown_connections 2026-02-24T18:38:15.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 -- 10.20.193.18:0/4119445972 wait complete. 2026-02-24T18:38:15.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:15.345+0000 7f969ec2b640 1 librados: shutdown 2026-02-24T18:38:18.357 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:18.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.424+0000 7fe12553e640 10 monclient: get_monmap_and_config 2026-02-24T18:38:18.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.424+0000 7fe12553e640 10 monclient: build_initial_monmap 2026-02-24T18:38:18.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.424+0000 7fe12553e640 10 monclient: monmap: 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:18.425736+0000 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:18.425736+0000 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:18.427 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:18.428 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:18.428 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 Processor -- start 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 -- start start 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient: init 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient: _add_conns ranks=[5,0,4,2,1,3] 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe12006ff20 0x7fe120070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-c con 0x7fe12006ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe1200c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe12015a5e0 0x7fe12014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-b con 0x7fe12015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1200c7e90 con 0x7fe1200c99a0 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe120159a40 con 0x7fe12015a5e0 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe1200c8170 con 0x7fe12006ff20 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.425+0000 7fe12553e640 10 monclient(hunting): authenticate will time out at 1642.106955s 2026-02-24T18:38:18.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11e7fc640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fe1200c99a0 0x7fe120159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:54462/0 (socket says 10.20.193.18:54462) 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11e7fc640 1 -- 10.20.193.18:0/2688505809 learned_addr learned my addr 10.20.193.18:0/2688505809 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11f7fe640 1 --2- 10.20.193.18:0/2688505809 >> v2:10.20.193.32:3300/0 conn(0x7fe12015a5e0 0x7fe12014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11effd640 1 --2- 10.20.193.18:0/2688505809 >> v2:10.20.193.110:3300/0 conn(0x7fe12006ff20 0x7fe120070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11f7fe640 10 monclient(hunting): get_auth_request con 0x7fe12015a5e0 auth_method 0 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11effd640 10 monclient(hunting): get_auth_request con 0x7fe12006ff20 auth_method 0 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.426+0000 7fe11effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:18.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 1 -- 10.20.193.18:0/2688505809 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413704346 0 0) 0x7fe1200c7e90 con 0x7fe1200c99a0 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 10 monclient(hunting): _negotiate 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 10 monclient(hunting): my global_id is 4908 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.427+0000 7fe11dffb640 1 -- 10.20.193.18:0/2688505809 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe10c003620 con 0x7fe1200c99a0 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_done global_id 4900 payload 274 2026-02-24T18:38:18.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 1 -- 10.20.193.18:0/2688505809 >> v2:10.20.193.110:3300/0 conn(0x7fe12006ff20 msgr2=0x7fe120070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 1 --2- 10.20.193.18:0/2688505809 >> v2:10.20.193.110:3300/0 conn(0x7fe12006ff20 0x7fe120070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 1 -- 10.20.193.18:0/2688505809 >> v1:10.20.193.18:6789/0 conn(0x7fe1200c99a0 legacy=0x7fe120159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 1 monclient: found mon.noname-b 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.428+0000 7fe11f7fe640 1 -- 10.20.193.18:0/2688505809 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1200c8450 con 0x7fe12015a5e0 2026-02-24T18:38:18.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.429+0000 7fe11f7fe640 1 --2- 10.20.193.18:0/2688505809 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12015a5e0 0x7fe12014fe10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe110009d90 tx=0x7fe11002f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce683b7f8096d591 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:18.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 1 -- 10.20.193.18:0/2688505809 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11003c070 con 0x7fe12015a5e0 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: dump: 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.442 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: _finish_auth 0 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 1 -- 10.20.193.18:0/2688505809 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe110037440 con 0x7fe12015a5e0 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 5 monclient: authenticate success, global_id 4900 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 1 -- 10.20.193.18:0/2688505809 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110037810 con 0x7fe12015a5e0 2026-02-24T18:38:18.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe11dffb640 10 monclient: dump: 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 4 set_mon_vals no callback set 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 10 monclient: shutdown 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 -- 10.20.193.18:0/2688505809 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12015a5e0 msgr2=0x7fe12014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 --2- 10.20.193.18:0/2688505809 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12015a5e0 0x7fe12014fe10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe110009d90 tx=0x7fe11002f1e0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 -- 10.20.193.18:0/2688505809 shutdown_connections 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 --2- 10.20.193.18:0/2688505809 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12015a5e0 0x7fe12014fe10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 --2- 10.20.193.18:0/2688505809 >> v2:10.20.193.110:3300/0 conn(0x7fe12006ff20 0x7fe120070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 -- 10.20.193.18:0/2688505809 >> 10.20.193.18:0/2688505809 conn(0x7fe1200cd780 msgr2=0x7fe1200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 -- 10.20.193.18:0/2688505809 shutdown_connections 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.431+0000 7fe12553e640 1 -- 10.20.193.18:0/2688505809 wait complete. 2026-02-24T18:38:18.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 10 monclient: build_initial_monmap 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 10 monclient: monmap: 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 librados: starting msgr at 2026-02-24T18:38:18.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 librados: starting objecter 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 Processor -- start 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 -- start start 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 librados: setting wanted keys 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 1 librados: calling monclient init 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.432+0000 7fe12553e640 10 monclient: init 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12006ff20 0x7fe1200c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-b con 0x7fe12006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-a con 0x7fe1200c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:18.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe12015a5e0 0x7fe1200bad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): picked mon.noname-c con 0x7fe12015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe120150350 con 0x7fe1200c99a0 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200c8a10 con 0x7fe12006ff20 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201504d0 con 0x7fe12015a5e0 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe12553e640 10 monclient(hunting): authenticate will time out at 1642.114297s 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11effd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12006ff20 0x7fe1200c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:47070/0 (socket says 10.20.193.18:47070) 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 1 -- 10.20.193.18:0/1119811401 learned_addr learned my addr 10.20.193.18:0/1119811401 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): get_auth_request con 0x7fe1200c99a0 auth_method 0 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11f7fe640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe12015a5e0 0x7fe1200bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11effd640 10 monclient(hunting): get_auth_request con 0x7fe12006ff20 auth_method 0 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11f7fe640 10 monclient(hunting): get_auth_request con 0x7fe12015a5e0 auth_method 0 2026-02-24T18:38:18.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.433+0000 7fe11e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.434+0000 7fe11f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 10 monclient(hunting): handle_auth_done global_id 4911 payload 824 2026-02-24T18:38:18.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 -- 10.20.193.18:0/1119811401 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe12015a5e0 msgr2=0x7fe1200bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe12015a5e0 0x7fe1200bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 -- 10.20.193.18:0/1119811401 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12006ff20 msgr2=0x7fe1200c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12006ff20 0x7fe1200c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 monclient: found mon.noname-a 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.435+0000 7fe11e7fc640 1 -- 10.20.193.18:0/1119811401 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1200bb250 con 0x7fe1200c99a0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.436+0000 7fe11e7fc640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c730 tx=0x7fe10800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.436+0000 7fe0ff7fe640 10 client.?.objecter ms_handle_connect 0x7fe1200c99a0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.436+0000 7fe0ff7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108017070 con 0x7fe1200c99a0 2026-02-24T18:38:18.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 10 monclient: dump: 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:18.452 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 10 monclient: _finish_auth 0 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe0ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 5 monclient: authenticate success, global_id 4911 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 10 monclient: _renew_subs 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1200bb4e0 con 0x7fe1200c99a0 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 10 client.4911.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 10 monclient: _renew_subs 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1200bbaa0 con 0x7fe1200c99a0 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.437+0000 7fe12553e640 1 librados: init done 2026-02-24T18:38:18.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe12553e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe12553e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe12553e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe120150860 con 0x7fe1200c99a0 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108005bb0 con 0x7fe1200c99a0 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108021060 con 0x7fe1200c99a0 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe11d7fa640 4 set_mon_vals no callback set 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.438+0000 7fe0ff7fe640 10 monclient: dump: 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:18.454 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:18.455 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:18.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.546+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe108024080 con 0x7fe1200c99a0 2026-02-24T18:38:18.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.546+0000 7fe0ff7fe640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 0x7fe0ec07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:18.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.546+0000 7fe11effd640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 0x7fe0ec07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:18.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.546+0000 7fe11effd640 10 monclient: get_auth_request con 0x7fe0ec078e00 auth_method 0 2026-02-24T18:38:18.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.547+0000 7fe11effd640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 0x7fe0ec07b2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c40 tx=0x7fe114009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:18.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.547+0000 7fe0ff7fe640 10 client.4911.objecter ms_handle_connect 0x7fe0ec078e00 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fe10805d4f0 con 0x7fe1200c99a0 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 10 client.4911.objecter ms_dispatch2 0x7fe120000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 3 client.4911.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 3 client.4911.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 20 client.4911.objecter dump_active .. 0 homeless 2026-02-24T18:38:18.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe1080614b0 con 0x7fe1200c99a0 2026-02-24T18:38:18.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:18.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.553+0000 7fe0ff7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:18.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.652+0000 7fe12553e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:18.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.652+0000 7fe12553e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:18.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.652+0000 7fe12553e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:18.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.652+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe1200c7410 con 0x7fe1200c99a0 2026-02-24T18:38:18.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.655+0000 7fe0ff7fe640 1 -- 10.20.193.18:0/1119811401 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fe1080657b0 con 0x7fe1200c99a0 2026-02-24T18:38:18.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.655+0000 7fe0ff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:18.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.655+0000 7fe0ff7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:18.657 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:18.657 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 10 librados: watch_flush enter 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 10 librados: watch_flush exit 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 20 client.4911.objecter shutdown clearing up homeless session... 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 10 client.4911.objecter successfully canceled tick 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 msgr2=0x7fe0ec07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 0x7fe0ec07b2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c40 tx=0x7fe114009290 comp rx=0 tx=0).stop 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 10 monclient: shutdown 2026-02-24T18:38:18.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 msgr2=0x7fe1200bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c730 tx=0x7fe10800cc00 comp rx=0 tx=0).stop 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 shutdown_connections 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe0ec078e00 0x7fe0ec07b2c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe12015a5e0 0x7fe1200bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe1200c99a0 0x7fe1200bd1f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 --2- 10.20.193.18:0/1119811401 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe12006ff20 0x7fe1200c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 >> 10.20.193.18:0/1119811401 conn(0x7fe1200cd780 msgr2=0x7fe1200cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 shutdown_connections 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 -- 10.20.193.18:0/1119811401 wait complete. 2026-02-24T18:38:18.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:18.656+0000 7fe12553e640 1 librados: shutdown 2026-02-24T18:38:19.542 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:57.488 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.544 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:57.490 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.551 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:04:57.497 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.570 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:57.516 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.571 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:57.517 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.573 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:57.482 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.575 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:57.482 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.579 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:04:57.525 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:19.582 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:04:57.491 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:21.669 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.736+0000 7f350b955640 10 monclient: get_monmap_and_config 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.736+0000 7f350b955640 10 monclient: build_initial_monmap 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient: monmap: 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:21.738861+0000 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:21.738861+0000 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:21.740 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 1 Processor -- start 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 1 -- start start 2026-02-24T18:38:21.741 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient: init 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient: _add_conns ranks=[3,2,0,1,4,5] 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f350406ff20 0x7f3504070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient(hunting): picked mon.noname-a con 0x7f350406ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f35040c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.737+0000 7f350b955640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f350415a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:21.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35040c7e90 con 0x7f350415a5e0 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35040c8170 con 0x7f35040c99a0 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3504159a40 con 0x7f350406ff20 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f350b955640 10 monclient(hunting): authenticate will time out at 1645.420021s 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f350406ff20 0x7f3504070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f3508ec9640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f35040c99a0 0x7f3504159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:36464/0 (socket says 10.20.193.18:36464) 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f3508ec9640 1 -- 10.20.193.18:0/3502826362 learned_addr learned my addr 10.20.193.18:0/3502826362 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): get_auth_request con 0x7f350406ff20 auth_method 0 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:21.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.738+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.739+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 1 -- 10.20.193.18:0/3502826362 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247185898 0 0) 0x7f35040c8170 con 0x7f35040c99a0 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 10 monclient(hunting): _negotiate 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 10 monclient(hunting): my global_id is 4877 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f34f3fff640 1 -- 10.20.193.18:0/3502826362 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34ec003620 con 0x7f35040c99a0 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 10 monclient(hunting): handle_auth_done global_id 4914 payload 274 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 1 -- 10.20.193.18:0/3502826362 >> v1:10.20.193.110:6789/0 conn(0x7f35040c99a0 legacy=0x7f3504159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 1 -- 10.20.193.18:0/3502826362 >> v1:10.20.193.18:6789/0 conn(0x7f350415a5e0 legacy=0x7f350414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:21.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 10 monclient: _finish_hunting 0 2026-02-24T18:38:21.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 1 monclient: found mon.noname-a 2026-02-24T18:38:21.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:21.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:21.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.740+0000 7f35096ca640 1 -- 10.20.193.18:0/3502826362 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040c8450 con 0x7f350406ff20 2026-02-24T18:38:21.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.741+0000 7f35096ca640 1 --2- 10.20.193.18:0/3502826362 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504070300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098d0 tx=0x7f34f402ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c5a96004e53bf server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:21.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 1 -- 10.20.193.18:0/3502826362 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f403c070 con 0x7f350406ff20 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: dump: 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:21.754 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: _finish_auth 0 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: _check_auth_tickets 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 1 -- 10.20.193.18:0/3502826362 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f402fb00 con 0x7f350406ff20 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 5 monclient: authenticate success, global_id 4914 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 1 -- 10.20.193.18:0/3502826362 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4037440 con 0x7f350406ff20 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 4 set_mon_vals no callback set 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:21.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f34f3fff640 10 monclient: dump: 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 10 monclient: shutdown 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:21.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 -- 10.20.193.18:0/3502826362 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 msgr2=0x7f3504070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 --2- 10.20.193.18:0/3502826362 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504070300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098d0 tx=0x7f34f402ee80 comp rx=0 tx=0).stop 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 -- 10.20.193.18:0/3502826362 shutdown_connections 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 --2- 10.20.193.18:0/3502826362 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504070300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 -- 10.20.193.18:0/3502826362 >> 10.20.193.18:0/3502826362 conn(0x7f35040cd780 msgr2=0x7f35040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 -- 10.20.193.18:0/3502826362 shutdown_connections 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.743+0000 7f350b955640 1 -- 10.20.193.18:0/3502826362 wait complete. 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient: build_initial_monmap 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient: monmap: 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:21.757 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 librados: starting msgr at 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 librados: starting objecter 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 Processor -- start 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 -- start start 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 librados: setting wanted keys 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 librados: calling monclient init 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient: init 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:21.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): picked mon.noname-a con 0x7f350406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 0x7f35041f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): picked mon.noname-c con 0x7f35040c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f350415a5e0 0x7f35041f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): picked mon.noname-b con 0x7f350415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3504150520 con 0x7f350406ff20 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35040c8a10 con 0x7f350415a5e0 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35041506a0 con 0x7f35040c99a0 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:21.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.744+0000 7f350b955640 10 monclient(hunting): authenticate will time out at 1645.426896s 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 0x7f35041f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f350415a5e0 0x7f35041f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42842/0 (socket says 10.20.193.18:42842) 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 1 -- 10.20.193.18:0/3356578885 learned_addr learned my addr 10.20.193.18:0/3356578885 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): get_auth_request con 0x7f350406ff20 auth_method 0 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): get_auth_request con 0x7f35040c99a0 auth_method 0 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:21.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): get_auth_request con 0x7f350415a5e0 auth_method 0 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3508ec9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f35096ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.745+0000 7f3509ecb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 10 monclient(hunting): handle_auth_done global_id 4880 payload 824 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 -- 10.20.193.18:0/3356578885 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f350415a5e0 msgr2=0x7f35041f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f350415a5e0 0x7f35041f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 -- 10.20.193.18:0/3356578885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 msgr2=0x7f3504158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 10 monclient: _finish_hunting 0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 monclient: found mon.noname-c 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.747+0000 7f3508ec9640 1 -- 10.20.193.18:0/3356578885 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350414b690 con 0x7f35040c99a0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.748+0000 7f3508ec9640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 0x7f35041f3910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c530 tx=0x7f34f800ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.748+0000 7f34f1ffb640 10 client.?.objecter ms_handle_connect 0x7f35040c99a0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.748+0000 7f34f1ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8017070 con 0x7f35040c99a0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:21.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: dump: 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:38:21.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 5 monclient: authenticate success, global_id 4880 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 10 monclient: _renew_subs 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041f6eb0 con 0x7f35040c99a0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 10 client.4880.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 10 monclient: _renew_subs 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041f7470 con 0x7f35040c99a0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f350b955640 1 librados: init done 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f8005de0 con 0x7f35040c99a0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8007270 con 0x7f35040c99a0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:21.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f1ffb640 10 monclient: dump: 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.749+0000 7f34f37fe640 4 set_mon_vals no callback set 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.750+0000 7f350b955640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.750+0000 7f350b955640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.750+0000 7f350b955640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:21.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.750+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3504070300 con 0x7f35040c99a0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.761+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f34f8007410 con 0x7f35040c99a0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.761+0000 7f34f1ffb640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 0x7f34d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.761+0000 7f35096ca640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 0x7f34d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.761+0000 7f35096ca640 10 monclient: get_auth_request con 0x7f34d4078e00 auth_method 0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.762+0000 7f34f1ffb640 10 client.4880.objecter ms_handle_connect 0x7f34d4078e00 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.762+0000 7f35096ca640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 0x7f34d407b2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34f4002410 tx=0x7f34f4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.763+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f34f8096ed0 con 0x7f35040c99a0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.763+0000 7f34f1ffb640 10 client.4880.objecter ms_dispatch2 0x7f3504000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.763+0000 7f34f1ffb640 3 client.4880.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.763+0000 7f34f1ffb640 3 client.4880.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:21.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.763+0000 7f34f1ffb640 20 client.4880.objecter dump_active .. 0 homeless 2026-02-24T18:38:21.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.870+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f34f8022070 con 0x7f35040c99a0 2026-02-24T18:38:21.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.870+0000 7f34f1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:21.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.870+0000 7f34f1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:21.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.970+0000 7f350b955640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:21.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.970+0000 7f350b955640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:21.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.970+0000 7f350b955640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:21.973 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.970+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f35041573f0 con 0x7f35040c99a0 2026-02-24T18:38:21.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f34f1ffb640 1 -- 10.20.193.18:0/3356578885 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f350414b690 con 0x7f35040c99a0 2026-02-24T18:38:21.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f34f1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:21.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f34f1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:21.976 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:21.977 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 10 librados: watch_flush enter 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 10 librados: watch_flush exit 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 20 client.4880.objecter shutdown clearing up homeless session... 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 10 client.4880.objecter successfully canceled tick 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 msgr2=0x7f34d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 0x7f34d407b2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34f4002410 tx=0x7f34f4033000 comp rx=0 tx=0).stop 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 10 monclient: shutdown 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 msgr2=0x7f35041f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 0x7f35041f3910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c530 tx=0x7f34f800ca00 comp rx=0 tx=0).stop 2026-02-24T18:38:21.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 shutdown_connections 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f34d4078e00 0x7f34d407b2c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f350415a5e0 0x7f35041f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f35040c99a0 0x7f35041f3910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 --2- 10.20.193.18:0/3356578885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f350406ff20 0x7f3504158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 >> 10.20.193.18:0/3356578885 conn(0x7f35040cd780 msgr2=0x7f350406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 shutdown_connections 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 -- 10.20.193.18:0/3356578885 wait complete. 2026-02-24T18:38:21.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:21.975+0000 7f350b955640 1 librados: shutdown 2026-02-24T18:38:24.986 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:25.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.056+0000 7fbf7a09f640 10 monclient: get_monmap_and_config 2026-02-24T18:38:25.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.056+0000 7fbf7a09f640 10 monclient: build_initial_monmap 2026-02-24T18:38:25.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.057+0000 7fbf7a09f640 10 monclient: monmap: 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:25.057848+0000 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:25.057848+0000 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.060 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.057+0000 7fbf7a09f640 1 Processor -- start 2026-02-24T18:38:25.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.057+0000 7fbf7a09f640 1 -- start start 2026-02-24T18:38:25.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.057+0000 7fbf7a09f640 10 monclient: init 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.057+0000 7fbf7a09f640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient: _add_conns ranks=[1,5,2,4,3,0] 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fbf7406ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbf740c99a0 0x7fbf74070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-c con 0x7fbf740c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:25.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbf7415a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf740c7e90 con 0x7fbf7406ff20 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf740c8170 con 0x7fbf7415a5e0 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbf74159a40 con 0x7fbf740c99a0 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf7a09f640 10 monclient(hunting): authenticate will time out at 1648.739060s 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf72ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbf740c99a0 0x7fbf74070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf737fe640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fbf7406ff20 0x7fbf74159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:36732/0 (socket says 10.20.193.18:36732) 2026-02-24T18:38:25.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf737fe640 1 -- 10.20.193.18:0/1721196857 learned_addr learned my addr 10.20.193.18:0/1721196857 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf72ffd640 10 monclient(hunting): get_auth_request con 0x7fbf740c99a0 auth_method 0 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.058+0000 7fbf72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.059+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.059+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.059+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 1 -- 10.20.193.18:0/1721196857 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284276429 0 0) 0x7fbf740c7e90 con 0x7fbf7406ff20 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 10 monclient(hunting): _negotiate 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 10 monclient(hunting): my global_id is 4909 2026-02-24T18:38:25.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf727fc640 1 -- 10.20.193.18:0/1721196857 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf58003620 con 0x7fbf7406ff20 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_done global_id 4883 payload 274 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 1 -- 10.20.193.18:0/1721196857 >> v1:10.20.193.110:6789/0 conn(0x7fbf7415a5e0 legacy=0x7fbf7414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 1 -- 10.20.193.18:0/1721196857 >> v1:10.20.193.32:6789/0 conn(0x7fbf7406ff20 legacy=0x7fbf74159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 1 monclient: found mon.noname-c 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.060+0000 7fbf72ffd640 1 -- 10.20.193.18:0/1721196857 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740c8450 con 0x7fbf740c99a0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.061+0000 7fbf72ffd640 1 --2- 10.20.193.18:0/1721196857 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf740c99a0 0x7fbf74070300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf640098d0 tx=0x7fbf6402ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee97ae32962c148 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 1 -- 10.20.193.18:0/1721196857 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6403c070 con 0x7fbf740c99a0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:25.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: dump: 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: _finish_auth 0 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: _check_auth_tickets 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 1 -- 10.20.193.18:0/1721196857 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6402fb00 con 0x7fbf740c99a0 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf7a09f640 5 monclient: authenticate success, global_id 4883 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 1 -- 10.20.193.18:0/1721196857 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64037440 con 0x7fbf740c99a0 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf7a09f640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf7a09f640 4 set_mon_vals no callback set 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.063+0000 7fbf727fc640 10 monclient: dump: 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:25.080 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 10 monclient: shutdown 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 -- 10.20.193.18:0/1721196857 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf740c99a0 msgr2=0x7fbf74070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/1721196857 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf740c99a0 0x7fbf74070300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf640098d0 tx=0x7fbf6402ee80 comp rx=0 tx=0).stop 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 -- 10.20.193.18:0/1721196857 shutdown_connections 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/1721196857 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf740c99a0 0x7fbf74070300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 -- 10.20.193.18:0/1721196857 >> 10.20.193.18:0/1721196857 conn(0x7fbf740cd780 msgr2=0x7fbf740cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 -- 10.20.193.18:0/1721196857 shutdown_connections 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 1 -- 10.20.193.18:0/1721196857 wait complete. 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 10 monclient: build_initial_monmap 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.064+0000 7fbf7a09f640 10 monclient: monmap: 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:25.082 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 librados: starting msgr at 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 librados: starting objecter 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 Processor -- start 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 -- start start 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 librados: setting wanted keys 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 librados: calling monclient init 2026-02-24T18:38:25.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient: init 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-c con 0x7fbf7406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 0x7fbf74073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-a con 0x7fbf740c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbf7415a5e0 0x7fbf741f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): picked mon.noname-b con 0x7fbf7415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74150520 con 0x7fbf740c99a0 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:25.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740c8a10 con 0x7fbf7415a5e0 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741506a0 con 0x7fbf7406ff20 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.065+0000 7fbf7a09f640 10 monclient(hunting): authenticate will time out at 1648.746643s 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 0x7fbf74073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbf7415a5e0 0x7fbf741f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 0x7fbf74073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:42852/0 (socket says 10.20.193.18:42852) 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:50698/0 (socket says 10.20.193.18:50698) 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 1 -- 10.20.193.18:0/345597986 learned_addr learned my addr 10.20.193.18:0/345597986 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): get_auth_request con 0x7fbf740c99a0 auth_method 0 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:25.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): get_auth_request con 0x7fbf7406ff20 auth_method 0 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): get_auth_request con 0x7fbf7415a5e0 auth_method 0 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:25.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.066+0000 7fbf72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 10 monclient(hunting): handle_auth_done global_id 4886 payload 824 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 -- 10.20.193.18:0/345597986 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbf7415a5e0 msgr2=0x7fbf741f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbf7415a5e0 0x7fbf741f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 -- 10.20.193.18:0/345597986 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 msgr2=0x7fbf74073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 0x7fbf74073080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 10 monclient: _finish_hunting 0 2026-02-24T18:38:25.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 monclient: found mon.noname-c 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.067+0000 7fbf737fe640 1 -- 10.20.193.18:0/345597986 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741e6160 con 0x7fbf7406ff20 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.068+0000 7fbf737fe640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c530 tx=0x7fbf6000ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.068+0000 7fbf4ffff640 10 client.?.objecter ms_handle_connect 0x7fbf7406ff20 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.068+0000 7fbf4ffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf7406ff20 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: dump: 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:25.088 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: _finish_auth 0 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 5 monclient: authenticate success, global_id 4886 2026-02-24T18:38:25.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _renew_subs 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf740c1610 con 0x7fbf7406ff20 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 client.4886.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _renew_subs 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf740c3d20 con 0x7fbf7406ff20 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 1 librados: init done 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf60005de0 con 0x7fbf7406ff20 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60021060 con 0x7fbf7406ff20 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:25.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf71ffb640 4 set_mon_vals no callback set 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf4ffff640 10 monclient: dump: 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:25.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:25.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.070+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf741573f0 con 0x7fbf7406ff20 2026-02-24T18:38:25.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.082+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbf60024080 con 0x7fbf7406ff20 2026-02-24T18:38:25.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.082+0000 7fbf4ffff640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 0x7fbf4407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:25.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.082+0000 7fbf72ffd640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 0x7fbf4407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:25.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.082+0000 7fbf72ffd640 10 monclient: get_auth_request con 0x7fbf44078e00 auth_method 0 2026-02-24T18:38:25.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf72ffd640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 0x7fbf4407b2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402eec0 tx=0x7fbf64033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 10 client.4886.objecter ms_handle_connect 0x7fbf44078e00 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fbf60096d50 con 0x7fbf7406ff20 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 10 client.4886.objecter ms_dispatch2 0x7fbf74000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 3 client.4886.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 3 client.4886.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:25.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.083+0000 7fbf4ffff640 20 client.4886.objecter dump_active .. 0 homeless 2026-02-24T18:38:25.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.180+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbf600614d0 con 0x7fbf7406ff20 2026-02-24T18:38:25.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.180+0000 7fbf4ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:25.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.180+0000 7fbf4ffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:25.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.279+0000 7fbf7a09f640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:25.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.279+0000 7fbf7a09f640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:25.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.279+0000 7fbf7a09f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:25.281 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.279+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbf741500b0 con 0x7fbf7406ff20 2026-02-24T18:38:25.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.281+0000 7fbf4ffff640 1 -- 10.20.193.18:0/345597986 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbf600657d0 con 0x7fbf7406ff20 2026-02-24T18:38:25.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.281+0000 7fbf4ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:25.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.281+0000 7fbf4ffff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:25.284 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:25.284 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:25.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 10 librados: watch_flush enter 2026-02-24T18:38:25.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 10 librados: watch_flush exit 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 20 client.4886.objecter shutdown clearing up homeless session... 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 10 client.4886.objecter successfully canceled tick 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 msgr2=0x7fbf4407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 0x7fbf4407b2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402eec0 tx=0x7fbf64033000 comp rx=0 tx=0).stop 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 10 monclient: shutdown 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 msgr2=0x7fbf740c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c530 tx=0x7fbf6000ca00 comp rx=0 tx=0).stop 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 shutdown_connections 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbf44078e00 0x7fbf4407b2c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbf7415a5e0 0x7fbf741f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbf740c99a0 0x7fbf74073080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 --2- 10.20.193.18:0/345597986 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbf7406ff20 0x7fbf740c10d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 >> 10.20.193.18:0/345597986 conn(0x7fbf740cd780 msgr2=0x7fbf740c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 shutdown_connections 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 -- 10.20.193.18:0/345597986 wait complete. 2026-02-24T18:38:25.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:25.283+0000 7fbf7a09f640 1 librados: shutdown 2026-02-24T18:38:28.294 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:28.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 10 monclient: get_monmap_and_config 2026-02-24T18:38:28.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 10 monclient: build_initial_monmap 2026-02-24T18:38:28.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 10 monclient: monmap: 2026-02-24T18:38:28.364 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:28.364 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:28.363121+0000 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:28.363121+0000 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 1 Processor -- start 2026-02-24T18:38:28.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 1 -- start start 2026-02-24T18:38:28.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.362+0000 7f6aa3271640 10 monclient: init 2026-02-24T18:38:28.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:28.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient: _add_conns ranks=[5,0,1,4,3,2] 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f6a9c0c4910 0x7f6a9c0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-c con 0x7f6a9c0c4910 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6a9c15ced0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6a9c15be80 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c15d630 con 0x7f6a9c15ced0 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c15d910 con 0x7f6a9c15be80 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9c15dbf0 con 0x7f6a9c0c4910 2026-02-24T18:38:28.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa3271640 10 monclient(hunting): authenticate will time out at 1652.044419s 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa1a6e640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f6a9c15ced0 0x7f6a9c15b770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:50226/0 (socket says 10.20.193.18:50226) 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa1a6e640 1 -- 10.20.193.18:0/1012643437 learned_addr learned my addr 10.20.193.18:0/1012643437 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.363+0000 7f6aa226f640 1 --2- 10.20.193.18:0/1012643437 >> v2:10.20.193.110:3300/0 conn(0x7f6a9c0c4910 0x7f6a9c0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): get_auth_request con 0x7f6a9c0c4910 auth_method 0 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.364+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:28.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 1 -- 10.20.193.18:0/1012643437 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3222765978 0 0) 0x7f6a9c15d630 con 0x7f6a9c15ced0 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 10 monclient(hunting): _negotiate 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 10 monclient(hunting): my global_id is 4923 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa126d640 1 -- 10.20.193.18:0/1012643437 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a84003620 con 0x7f6a9c15ced0 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 10 monclient(hunting): handle_auth_done global_id 4889 payload 274 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 1 -- 10.20.193.18:0/1012643437 >> v1:10.20.193.32:6789/0 conn(0x7f6a9c15be80 legacy=0x7f6a9c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 1 -- 10.20.193.18:0/1012643437 >> v1:10.20.193.18:6789/0 conn(0x7f6a9c15ced0 legacy=0x7f6a9c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 10 monclient: _finish_hunting 0 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 1 monclient: found mon.noname-c 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.365+0000 7f6aa226f640 1 -- 10.20.193.18:0/1012643437 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c15af70 con 0x7f6a9c0c4910 2026-02-24T18:38:28.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.366+0000 7f6aa226f640 1 --2- 10.20.193.18:0/1012643437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c0c4cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0098a0 tx=0x7f6a8c02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43df12b6c5101878 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 1 -- 10.20.193.18:0/1012643437 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c03c070 con 0x7f6a9c0c4910 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: dump: 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.378 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: _finish_auth 0 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: _check_auth_tickets 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:28.379 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 1 -- 10.20.193.18:0/1012643437 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8c02fb70 con 0x7f6a9c0c4910 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 5 monclient: authenticate success, global_id 4889 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 1 -- 10.20.193.18:0/1012643437 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c037440 con 0x7f6a9c0c4910 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 4 set_mon_vals no callback set 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa126d640 10 monclient: dump: 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.380 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 10 monclient: shutdown 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 -- 10.20.193.18:0/1012643437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 msgr2=0x7f6a9c0c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 --2- 10.20.193.18:0/1012643437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c0c4cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0098a0 tx=0x7f6a8c02ee60 comp rx=0 tx=0).stop 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 -- 10.20.193.18:0/1012643437 shutdown_connections 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 --2- 10.20.193.18:0/1012643437 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c0c4cf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.381 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 -- 10.20.193.18:0/1012643437 >> 10.20.193.18:0/1012643437 conn(0x7f6a9c0cd480 msgr2=0x7f6a9c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 -- 10.20.193.18:0/1012643437 shutdown_connections 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.368+0000 7f6aa3271640 1 -- 10.20.193.18:0/1012643437 wait complete. 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient: build_initial_monmap 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient: monmap: 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:28.382 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 librados: starting msgr at 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 librados: starting objecter 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 Processor -- start 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 -- start start 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 librados: setting wanted keys 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 librados: calling monclient init 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient: init 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:28.383 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-c con 0x7f6a9c0c4910 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a9c15be80 0x7f6a9c06eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-a con 0x7f6a9c15be80 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a9c15ced0 0x7f6a9c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): picked mon.noname-b con 0x7f6a9c15ced0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c153ab0 con 0x7f6a9c15be80 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c15d340 con 0x7f6a9c15ced0 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.369+0000 7f6aa3271640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa3271640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c153c30 con 0x7f6a9c0c4910 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa3271640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa3271640 10 monclient(hunting): authenticate will time out at 1652.051006s 2026-02-24T18:38:28.384 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a9c15be80 0x7f6a9c06eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:50712/0 (socket says 10.20.193.18:50712) 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 1 -- 10.20.193.18:0/2563738096 learned_addr learned my addr 10.20.193.18:0/2563738096 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): get_auth_request con 0x7f6a9c0c4910 auth_method 0 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): get_auth_request con 0x7f6a9c15be80 auth_method 0 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:28.385 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa2a70640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a9c15ced0 0x7f6a9c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa226f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa2a70640 10 monclient(hunting): get_auth_request con 0x7f6a9c15ced0 auth_method 0 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa2a70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa2a70640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa2a70640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.370+0000 7f6aa1a6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa2a70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa2a70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:28.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa2a70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 10 monclient(hunting): handle_auth_done global_id 4892 payload 824 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 -- 10.20.193.18:0/2563738096 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a9c15ced0 msgr2=0x7f6a9c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a9c15ced0 0x7f6a9c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 -- 10.20.193.18:0/2563738096 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a9c15be80 msgr2=0x7f6a9c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a9c15be80 0x7f6a9c06eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 10 monclient: _finish_hunting 0 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 monclient: found mon.noname-c 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.371+0000 7f6aa226f640 1 -- 10.20.193.18:0/2563738096 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c1f3480 con 0x7f6a9c0c4910 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.372+0000 7f6aa226f640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009130 tx=0x7f6a8c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:28.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.372+0000 7f6a82ffd640 10 client.?.objecter ms_handle_connect 0x7f6a9c0c4910 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.372+0000 7f6a82ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c03c040 con 0x7f6a9c0c4910 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: dump: 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:28.388 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: _finish_auth 0 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8c042630 con 0x7f6a9c0c4910 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c042a20 con 0x7f6a9c0c4910 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:28.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa0a6c640 4 set_mon_vals no callback set 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6a82ffd640 10 monclient: dump: 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:28.390 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 5 monclient: authenticate success, global_id 4892 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 10 monclient: _renew_subs 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c1f3760 con 0x7f6a9c0c4910 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 10 client.4892.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 10 monclient: _renew_subs 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c1f3d20 con 0x7f6a9c0c4910 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.374+0000 7f6aa3271640 1 librados: init done 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.375+0000 7f6aa3271640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.375+0000 7f6aa3271640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.375+0000 7f6aa3271640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.375+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c0c4cf0 con 0x7f6a9c0c4910 2026-02-24T18:38:28.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.479+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6a8c04b450 con 0x7f6a9c0c4910 2026-02-24T18:38:28.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.479+0000 7f6a82ffd640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 0x7f6a6c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:28.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.480+0000 7f6aa1a6e640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 0x7f6a6c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:28.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.480+0000 7f6aa1a6e640 10 monclient: get_auth_request con 0x7f6a6c078e00 auth_method 0 2026-02-24T18:38:28.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.480+0000 7f6aa1a6e640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 0x7f6a6c07b2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c15a7e0 tx=0x7f6a90007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:28.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.480+0000 7f6a82ffd640 10 client.4892.objecter ms_handle_connect 0x7f6a6c078e00 2026-02-24T18:38:28.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.481+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f6a8c02fab0 con 0x7f6a9c0c4910 2026-02-24T18:38:28.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.481+0000 7f6a82ffd640 10 client.4892.objecter ms_dispatch2 0x7f6a9c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:28.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.481+0000 7f6a82ffd640 3 client.4892.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:28.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.481+0000 7f6a82ffd640 3 client.4892.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:28.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.481+0000 7f6a82ffd640 20 client.4892.objecter dump_active .. 0 homeless 2026-02-24T18:38:28.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.487+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6a8c086990 con 0x7f6a9c0c4910 2026-02-24T18:38:28.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.487+0000 7f6a82ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:28.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.487+0000 7f6a82ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:28.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.588+0000 7f6aa3271640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:28.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.588+0000 7f6aa3271640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:28.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.588+0000 7f6aa3271640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:28.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.588+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a9c150fd0 con 0x7f6a9c0c4910 2026-02-24T18:38:28.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.591+0000 7f6a82ffd640 1 -- 10.20.193.18:0/2563738096 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6a8c08be20 con 0x7f6a9c0c4910 2026-02-24T18:38:28.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.591+0000 7f6a82ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:28.593 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.591+0000 7f6a82ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:28.593 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:28.594 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:28.594 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 10 librados: watch_flush enter 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 10 librados: watch_flush exit 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 20 client.4892.objecter shutdown clearing up homeless session... 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 10 client.4892.objecter successfully canceled tick 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 msgr2=0x7f6a6c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 0x7f6a6c07b2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c15a7e0 tx=0x7f6a90007dd0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 10 monclient: shutdown 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 msgr2=0x7f6a9c1f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009130 tx=0x7f6a8c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 shutdown_connections 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6a6c078e00 0x7f6a6c07b2c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f6a9c15ced0 0x7f6a9c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f6a9c15be80 0x7f6a9c06eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 --2- 10.20.193.18:0/2563738096 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f6a9c0c4910 0x7f6a9c1f7260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.592+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 >> 10.20.193.18:0/2563738096 conn(0x7f6a9c0cd480 msgr2=0x7f6a9c14f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.593+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 shutdown_connections 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.593+0000 7f6aa3271640 1 -- 10.20.193.18:0/2563738096 wait complete. 2026-02-24T18:38:28.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:28.593+0000 7f6aa3271640 1 librados: shutdown 2026-02-24T18:38:31.604 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:31.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 10 monclient: get_monmap_and_config 2026-02-24T18:38:31.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 10 monclient: build_initial_monmap 2026-02-24T18:38:31.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 10 monclient: monmap: 2026-02-24T18:38:31.672 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:31.671170+0000 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:31.671170+0000 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:31.673 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 1 Processor -- start 2026-02-24T18:38:31.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 1 -- start start 2026-02-24T18:38:31.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.670+0000 7fe68ed76640 10 monclient: init 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient: _add_conns ranks=[4,0,1,5,2,3] 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe68806ff20 0x7fe688070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-b con 0x7fe68806ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6880c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:31.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe68815a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6880c7e90 con 0x7fe6880c99a0 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6880c8170 con 0x7fe68815a5e0 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe688159a40 con 0x7fe68806ff20 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68ed76640 10 monclient(hunting): authenticate will time out at 1655.352346s 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe68caeb640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe68806ff20 0x7fe688070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe67ffff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fe6880c99a0 0x7fe688159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:50006/0 (socket says 10.20.193.18:50006) 2026-02-24T18:38:31.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.671+0000 7fe67ffff640 1 -- 10.20.193.18:0/933745621 learned_addr learned my addr 10.20.193.18:0/933745621 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): get_auth_request con 0x7fe68806ff20 auth_method 0 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.672+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 1 -- 10.20.193.18:0/933745621 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091047738 0 0) 0x7fe6880c7e90 con 0x7fe6880c99a0 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 10 monclient(hunting): my global_id is 4929 2026-02-24T18:38:31.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe67f7fe640 1 -- 10.20.193.18:0/933745621 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe674003620 con 0x7fe6880c99a0 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 10 monclient(hunting): handle_auth_done global_id 4921 payload 274 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 1 -- 10.20.193.18:0/933745621 >> v1:10.20.193.32:6789/0 conn(0x7fe68815a5e0 legacy=0x7fe68814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 1 -- 10.20.193.18:0/933745621 >> v1:10.20.193.18:6789/0 conn(0x7fe6880c99a0 legacy=0x7fe688159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 10 monclient: _finish_hunting 0 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 1 monclient: found mon.noname-b 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.673+0000 7fe68caeb640 1 -- 10.20.193.18:0/933745621 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6880c8450 con 0x7fe68806ff20 2026-02-24T18:38:31.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.674+0000 7fe68caeb640 1 --2- 10.20.193.18:0/933745621 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe68806ff20 0x7fe688070300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6780098d0 tx=0x7fe67802ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9de978081e24eac server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:31.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 1 -- 10.20.193.18:0/933745621 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67803c070 con 0x7fe68806ff20 2026-02-24T18:38:31.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:31.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:31.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: dump: 2026-02-24T18:38:31.686 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:38:31.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 1 -- 10.20.193.18:0/933745621 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67802fb00 con 0x7fe68806ff20 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 5 monclient: authenticate success, global_id 4921 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 1 -- 10.20.193.18:0/933745621 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678037440 con 0x7fe68806ff20 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 4 set_mon_vals no callback set 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe67f7fe640 10 monclient: dump: 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:31.688 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 10 monclient: shutdown 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 1 -- 10.20.193.18:0/933745621 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe68806ff20 msgr2=0x7fe688070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.676+0000 7fe68ed76640 1 --2- 10.20.193.18:0/933745621 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe68806ff20 0x7fe688070300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6780098d0 tx=0x7fe67802ee80 comp rx=0 tx=0).stop 2026-02-24T18:38:31.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 -- 10.20.193.18:0/933745621 shutdown_connections 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 --2- 10.20.193.18:0/933745621 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe68806ff20 0x7fe688070300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 -- 10.20.193.18:0/933745621 >> 10.20.193.18:0/933745621 conn(0x7fe6880cd780 msgr2=0x7fe6880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 -- 10.20.193.18:0/933745621 shutdown_connections 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 -- 10.20.193.18:0/933745621 wait complete. 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 10 monclient: build_initial_monmap 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 10 monclient: monmap: 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:31.690 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 librados: starting msgr at 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 librados: starting objecter 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 Processor -- start 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 -- start start 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 librados: setting wanted keys 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 1 librados: calling monclient init 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.677+0000 7fe68ed76640 10 monclient: init 2026-02-24T18:38:31.691 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-a con 0x7fe68806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 0x7fe6881f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-b con 0x7fe6880c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe68815a5e0 0x7fe688154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): picked mon.noname-c con 0x7fe68815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe688150520 con 0x7fe68806ff20 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880c8a10 con 0x7fe6880c99a0 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:31.692 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6881506a0 con 0x7fe68815a5e0 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68ed76640 10 monclient(hunting): authenticate will time out at 1655.359221s 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 0x7fe6881f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45148/0 (socket says 10.20.193.18:45148) 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 1 -- 10.20.193.18:0/4192709826 learned_addr learned my addr 10.20.193.18:0/4192709826 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): get_auth_request con 0x7fe68806ff20 auth_method 0 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68d2ec640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe68815a5e0 0x7fe688154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:31.693 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): get_auth_request con 0x7fe6880c99a0 auth_method 0 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe68caeb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.678+0000 7fe67ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe67ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): get_auth_request con 0x7fe68815a5e0 auth_method 0 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:31.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.679+0000 7fe68d2ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 10 monclient(hunting): handle_auth_done global_id 4924 payload 824 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 -- 10.20.193.18:0/4192709826 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe68815a5e0 msgr2=0x7fe688154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe68815a5e0 0x7fe688154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 -- 10.20.193.18:0/4192709826 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 msgr2=0x7fe6881f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 monclient: found mon.noname-b 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe67ffff640 1 -- 10.20.193.18:0/4192709826 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68806d700 con 0x7fe6880c99a0 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.680+0000 7fe68caeb640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.681+0000 7fe67ffff640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 0x7fe6881f6f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c500 tx=0x7fe67000c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.681+0000 7fe67d7fa640 10 client.?.objecter ms_handle_connect 0x7fe6880c99a0 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.681+0000 7fe67d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670017070 con 0x7fe6880c99a0 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: dump: 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.696 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 5 monclient: authenticate success, global_id 4924 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 10 monclient: _renew_subs 2026-02-24T18:38:31.697 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68806d9e0 con 0x7fe6880c99a0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 10 client.4924.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 10 monclient: _renew_subs 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68806dfa0 con 0x7fe6880c99a0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe68ed76640 1 librados: init done 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670005de0 con 0x7fe6880c99a0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67effd640 4 set_mon_vals no callback set 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670021060 con 0x7fe6880c99a0 2026-02-24T18:38:31.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.682+0000 7fe67d7fa640 10 monclient: dump: 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:31.699 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:31.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.683+0000 7fe68ed76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:31.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.683+0000 7fe68ed76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:31.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.683+0000 7fe68ed76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.683+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe688070300 con 0x7fe6880c99a0 2026-02-24T18:38:31.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.695+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe670024080 con 0x7fe6880c99a0 2026-02-24T18:38:31.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.695+0000 7fe67d7fa640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 0x7fe65807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.695+0000 7fe68caeb640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 0x7fe65807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.696+0000 7fe68caeb640 10 monclient: get_auth_request con 0x7fe658078e00 auth_method 0 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.696+0000 7fe68caeb640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 0x7fe65807b2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe678002410 tx=0x7fe678033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.696+0000 7fe67d7fa640 10 client.4924.objecter ms_handle_connect 0x7fe658078e00 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.697+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fe670096cd0 con 0x7fe6880c99a0 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.697+0000 7fe67d7fa640 10 client.4924.objecter ms_dispatch2 0x7fe688000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.697+0000 7fe67d7fa640 3 client.4924.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.697+0000 7fe67d7fa640 3 client.4924.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:31.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.697+0000 7fe67d7fa640 20 client.4924.objecter dump_active .. 0 homeless 2026-02-24T18:38:31.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.798+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe670061560 con 0x7fe6880c99a0 2026-02-24T18:38:31.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.798+0000 7fe67d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:31.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.798+0000 7fe67d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:31.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.898+0000 7fe68ed76640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:31.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.898+0000 7fe68ed76640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:31.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.898+0000 7fe68ed76640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:31.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.898+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe68814f440 con 0x7fe6880c99a0 2026-02-24T18:38:31.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.900+0000 7fe67d7fa640 1 -- 10.20.193.18:0/4192709826 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fe670065860 con 0x7fe6880c99a0 2026-02-24T18:38:31.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.900+0000 7fe67d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:31.902 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.900+0000 7fe67d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:31.902 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:31.902 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 10 librados: watch_flush enter 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 10 librados: watch_flush exit 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 20 client.4924.objecter shutdown clearing up homeless session... 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 10 client.4924.objecter successfully canceled tick 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 msgr2=0x7fe65807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 0x7fe65807b2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe678002410 tx=0x7fe678033000 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 10 monclient: shutdown 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 msgr2=0x7fe6881f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 0x7fe6881f6f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c500 tx=0x7fe67000c9d0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 shutdown_connections 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe658078e00 0x7fe65807b2c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe6880c99a0 0x7fe6881f6f10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe68815a5e0 0x7fe688154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 --2- 10.20.193.18:0/4192709826 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe68806ff20 0x7fe6881f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 >> 10.20.193.18:0/4192709826 conn(0x7fe6880cd780 msgr2=0x7fe68814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 shutdown_connections 2026-02-24T18:38:31.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 -- 10.20.193.18:0/4192709826 wait complete. 2026-02-24T18:38:31.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:31.901+0000 7fe68ed76640 1 librados: shutdown 2026-02-24T18:38:34.914 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:34.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.983+0000 7fda18a88640 10 monclient: get_monmap_and_config 2026-02-24T18:38:34.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.983+0000 7fda18a88640 10 monclient: build_initial_monmap 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.983+0000 7fda18a88640 10 monclient: monmap: 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:34.984748+0000 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:34.984748+0000 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:34.986 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:34.987 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:34.987 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 Processor -- start 2026-02-24T18:38:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 -- start start 2026-02-24T18:38:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient: init 2026-02-24T18:38:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:34.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient: _add_conns ranks=[3,5,0,1,4,2] 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fda1406ff20 0x7fda14070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-a con 0x7fda1406ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 0x7fda14159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-c con 0x7fda140c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda1415a410 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1415a860 con 0x7fda1415a410 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fda14159a40 con 0x7fda1406ff20 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fda140c7e90 con 0x7fda140c99a0 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:34.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:34.984+0000 7fda18a88640 10 monclient(hunting): authenticate will time out at 1658.665953s 2026-02-24T18:38:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.006+0000 7fda11d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 0x7fda14159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 0x7fda14159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:48908/0 (socket says 10.20.193.18:48908) 2026-02-24T18:38:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 1 -- 10.20.193.18:0/726726115 learned_addr learned my addr 10.20.193.18:0/726726115 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): get_auth_request con 0x7fda140c99a0 auth_method 0 2026-02-24T18:38:35.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 1 --2- 10.20.193.18:0/726726115 >> v2:10.20.193.18:3300/0 conn(0x7fda1406ff20 0x7fda14070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): get_auth_request con 0x7fda1406ff20 auth_method 0 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:35.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.007+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:35.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 10 monclient(hunting): handle_auth_done global_id 4935 payload 274 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 1 -- 10.20.193.18:0/726726115 >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 msgr2=0x7fda14159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 1 --2- 10.20.193.18:0/726726115 >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 0x7fda14159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 1 -- 10.20.193.18:0/726726115 >> v1:10.20.193.18:6789/0 conn(0x7fda1415a410 legacy=0x7fda1414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 10 monclient: _finish_hunting 0 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 1 monclient: found mon.noname-a 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.008+0000 7fda12575640 1 -- 10.20.193.18:0/726726115 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda140c8170 con 0x7fda1406ff20 2026-02-24T18:38:35.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.011+0000 7fda12575640 1 --2- 10.20.193.18:0/726726115 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14070300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda080098a0 tx=0x7fda0802ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358671bfc512dbb4 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 1 -- 10.20.193.18:0/726726115 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0803c070 con 0x7fda1406ff20 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: dump: 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.020 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: _finish_auth 0 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: _check_auth_tickets 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 1 -- 10.20.193.18:0/726726115 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0802fae0 con 0x7fda1406ff20 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 5 monclient: authenticate success, global_id 4935 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:35.021 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 1 -- 10.20.193.18:0/726726115 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08037440 con 0x7fda1406ff20 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 4 set_mon_vals no callback set 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda11573640 10 monclient: dump: 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:35.022 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 10 monclient: shutdown 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 -- 10.20.193.18:0/726726115 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 msgr2=0x7fda14070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 --2- 10.20.193.18:0/726726115 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14070300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda080098a0 tx=0x7fda0802ee10 comp rx=0 tx=0).stop 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 -- 10.20.193.18:0/726726115 shutdown_connections 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 --2- 10.20.193.18:0/726726115 >> v2:10.20.193.110:3300/0 conn(0x7fda140c99a0 0x7fda14159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 --2- 10.20.193.18:0/726726115 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14070300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 -- 10.20.193.18:0/726726115 >> 10.20.193.18:0/726726115 conn(0x7fda140cd780 msgr2=0x7fda140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 -- 10.20.193.18:0/726726115 shutdown_connections 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.014+0000 7fda18a88640 1 -- 10.20.193.18:0/726726115 wait complete. 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 10 monclient: build_initial_monmap 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 10 monclient: monmap: 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:35.024 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 librados: starting msgr at 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 librados: starting objecter 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 Processor -- start 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 -- start start 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 librados: setting wanted keys 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 1 librados: calling monclient init 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.015+0000 7fda18a88640 10 monclient: init 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:35.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-a con 0x7fda1406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fda140c99a0 0x7fda1414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-c con 0x7fda140c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fda1415a410 0x7fda14072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): picked mon.noname-b con 0x7fda1415a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fda140c8a10 con 0x7fda1406ff20 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda140c8730 con 0x7fda1415a410 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda14150520 con 0x7fda140c99a0 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda18a88640 10 monclient(hunting): authenticate will time out at 1658.697109s 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45168/0 (socket says 10.20.193.18:45168) 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 1 -- 10.20.193.18:0/1241876732 learned_addr learned my addr 10.20.193.18:0/1241876732 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): get_auth_request con 0x7fda1406ff20 auth_method 0 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda11d74640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fda140c99a0 0x7fda1414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12d76640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fda1415a410 0x7fda14072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:35.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda11d74640 10 monclient(hunting): get_auth_request con 0x7fda140c99a0 auth_method 0 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda11d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda11d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda11d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12d76640 10 monclient(hunting): get_auth_request con 0x7fda1415a410 auth_method 0 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.016+0000 7fda12d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda11d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda12d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda12d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:35.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.017+0000 7fda12d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 10 monclient(hunting): handle_auth_done global_id 4941 payload 824 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 -- 10.20.193.18:0/1241876732 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fda140c99a0 msgr2=0x7fda1414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fda140c99a0 0x7fda1414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 -- 10.20.193.18:0/1241876732 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fda1415a410 msgr2=0x7fda14072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fda1415a410 0x7fda14072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 10 monclient: _finish_hunting 0 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 monclient: found mon.noname-a 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.029 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.018+0000 7fda12575640 1 -- 10.20.193.18:0/1241876732 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda141f47c0 con 0x7fda1406ff20 2026-02-24T18:38:35.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.019+0000 7fda12575640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda0802f340 tx=0x7fda080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:35.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.019+0000 7fd9faffd640 10 client.?.objecter ms_handle_connect 0x7fda1406ff20 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.019+0000 7fd9faffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.020+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0803c040 con 0x7fda1406ff20 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: dump: 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:35.032 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: _finish_auth 0 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda08042450 con 0x7fda1406ff20 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08042840 con 0x7fda1406ff20 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:35.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fd9faffd640 10 monclient: dump: 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda10d72640 4 set_mon_vals no callback set 2026-02-24T18:38:35.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 5 monclient: authenticate success, global_id 4941 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _renew_subs 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141543d0 con 0x7fda1406ff20 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 client.4941.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _renew_subs 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda14154990 con 0x7fda1406ff20 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 1 librados: init done 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.021+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda14070300 con 0x7fda1406ff20 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.032+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fda080429e0 con 0x7fda1406ff20 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.032+0000 7fd9faffd640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 0x7fd9e007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.033+0000 7fda11d74640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 0x7fd9e007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.033+0000 7fda11d74640 10 monclient: get_auth_request con 0x7fd9e0078e30 auth_method 0 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.033+0000 7fda11d74640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 0x7fd9e007b2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda1406f360 tx=0x7fd9fc009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.033+0000 7fd9faffd640 10 client.4941.objecter ms_handle_connect 0x7fd9e0078e30 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.035+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fda0802f9a0 con 0x7fda1406ff20 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.035+0000 7fd9faffd640 10 client.4941.objecter ms_dispatch2 0x7fda14000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:35.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.035+0000 7fd9faffd640 3 client.4941.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:35.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.035+0000 7fd9faffd640 3 client.4941.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:35.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.035+0000 7fd9faffd640 20 client.4941.objecter dump_active .. 0 homeless 2026-02-24T18:38:35.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.136+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fda08087530 con 0x7fda1406ff20 2026-02-24T18:38:35.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.136+0000 7fd9faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:35.138 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.136+0000 7fd9faffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:35.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.237+0000 7fda18a88640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:35.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.237+0000 7fda18a88640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:35.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.237+0000 7fda18a88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:35.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.237+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda141573f0 con 0x7fda1406ff20 2026-02-24T18:38:35.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.240+0000 7fd9faffd640 1 -- 10.20.193.18:0/1241876732 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fda0808b830 con 0x7fda1406ff20 2026-02-24T18:38:35.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.240+0000 7fd9faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:35.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.240+0000 7fd9faffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:35.242 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:35.242 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:35.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 10 librados: watch_flush enter 2026-02-24T18:38:35.243 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 10 librados: watch_flush exit 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 20 client.4941.objecter shutdown clearing up homeless session... 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 10 client.4941.objecter successfully canceled tick 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 msgr2=0x7fd9e007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 0x7fd9e007b2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda1406f360 tx=0x7fd9fc009340 comp rx=0 tx=0).stop 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 10 monclient: shutdown 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 msgr2=0x7fda14155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda0802f340 tx=0x7fda080047c0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 shutdown_connections 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd9e0078e30 0x7fd9e007b2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fda1415a410 0x7fda14072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fda140c99a0 0x7fda1414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 --2- 10.20.193.18:0/1241876732 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fda1406ff20 0x7fda14155750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 >> 10.20.193.18:0/1241876732 conn(0x7fda140cd780 msgr2=0x7fda1406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 shutdown_connections 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 -- 10.20.193.18:0/1241876732 wait complete. 2026-02-24T18:38:35.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:35.241+0000 7fda18a88640 1 librados: shutdown 2026-02-24T18:38:38.252 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:38.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.321+0000 7fe53a5f7640 10 monclient: get_monmap_and_config 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.321+0000 7fe53a5f7640 10 monclient: build_initial_monmap 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.321+0000 7fe53a5f7640 10 monclient: monmap: 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:38.322416+0000 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:38.322416+0000 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:38.324 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 Processor -- start 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 -- start start 2026-02-24T18:38:38.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient: init 2026-02-24T18:38:38.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient: _add_conns ranks=[0,4,3,1,2,5] 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe53406ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe5340c99a0 0x7fe534070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-b con 0x7fe5340c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe53415a5e0 0x7fe53414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-a con 0x7fe53415a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5340c7e90 con 0x7fe53406ff20 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe534159a40 con 0x7fe53415a5e0 2026-02-24T18:38:38.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5340c8170 con 0x7fe5340c99a0 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.322+0000 7fe53a5f7640 10 monclient(hunting): authenticate will time out at 1662.003640s 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe53415a5e0 0x7fe53414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe5340c99a0 0x7fe534070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe533fff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fe53406ff20 0x7fe534159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:50032/0 (socket says 10.20.193.18:50032) 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe533fff640 1 -- 10.20.193.18:0/291815076 learned_addr learned my addr 10.20.193.18:0/291815076 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): get_auth_request con 0x7fe5340c99a0 auth_method 0 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:38.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): get_auth_request con 0x7fe53415a5e0 auth_method 0 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.323+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 10 monclient(hunting): handle_auth_done global_id 4930 payload 274 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 1 -- 10.20.193.18:0/291815076 >> v2:10.20.193.18:3300/0 conn(0x7fe53415a5e0 msgr2=0x7fe53414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 1 --2- 10.20.193.18:0/291815076 >> v2:10.20.193.18:3300/0 conn(0x7fe53415a5e0 0x7fe53414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 1 -- 10.20.193.18:0/291815076 >> v1:10.20.193.18:6789/0 conn(0x7fe53406ff20 legacy=0x7fe534159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 10 monclient: _finish_hunting 0 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 1 monclient: found mon.noname-b 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.324+0000 7fe5337fe640 1 -- 10.20.193.18:0/291815076 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5340c8450 con 0x7fe5340c99a0 2026-02-24T18:38:38.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.325+0000 7fe5337fe640 1 --2- 10.20.193.18:0/291815076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe534070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe52402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf9e0cb4a469ed7 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:38.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 1 -- 10.20.193.18:0/291815076 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52403c070 con 0x7fe5340c99a0 2026-02-24T18:38:38.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:38.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:38.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: dump: 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: _finish_auth 0 2026-02-24T18:38:38.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 1 -- 10.20.193.18:0/291815076 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52402fae0 con 0x7fe5340c99a0 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe53a5f7640 5 monclient: authenticate success, global_id 4930 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 1 -- 10.20.193.18:0/291815076 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524037440 con 0x7fe5340c99a0 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe532ffd640 10 monclient: dump: 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.340 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe53a5f7640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.327+0000 7fe53a5f7640 4 set_mon_vals no callback set 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 10 monclient: shutdown 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 -- 10.20.193.18:0/291815076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 msgr2=0x7fe534070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/291815076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe534070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe52402ee10 comp rx=0 tx=0).stop 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 -- 10.20.193.18:0/291815076 shutdown_connections 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/291815076 >> v2:10.20.193.18:3300/0 conn(0x7fe53415a5e0 0x7fe53414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/291815076 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe534070300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 -- 10.20.193.18:0/291815076 >> 10.20.193.18:0/291815076 conn(0x7fe5340cd780 msgr2=0x7fe5340cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 -- 10.20.193.18:0/291815076 shutdown_connections 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 -- 10.20.193.18:0/291815076 wait complete. 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 10 monclient: build_initial_monmap 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 10 monclient: monmap: 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:38.342 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 librados: starting msgr at 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.328+0000 7fe53a5f7640 1 librados: starting objecter 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 Processor -- start 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 -- start start 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 librados: setting wanted keys 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 librados: calling monclient init 2026-02-24T18:38:38.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient: init 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 0x7fe5340cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-a con 0x7fe53406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe5340bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-b con 0x7fe5340c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe53415a5e0 0x7fe5340bca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): picked mon.noname-c con 0x7fe53415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe534150350 con 0x7fe53406ff20 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340c8a10 con 0x7fe5340c99a0 2026-02-24T18:38:38.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341504d0 con 0x7fe53415a5e0 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe53a5f7640 10 monclient(hunting): authenticate will time out at 1662.010468s 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 0x7fe5340cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe5337fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe5340bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe538b6d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe53415a5e0 0x7fe5340bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 0x7fe5340cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:45178/0 (socket says 10.20.193.18:45178) 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 1 -- 10.20.193.18:0/1646567609 learned_addr learned my addr 10.20.193.18:0/1646567609 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 10 monclient(hunting): get_auth_request con 0x7fe53406ff20 auth_method 0 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:38.345 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe533fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe5337fe640 10 monclient(hunting): get_auth_request con 0x7fe5340c99a0 auth_method 0 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe5337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe5337fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.329+0000 7fe5337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): get_auth_request con 0x7fe53415a5e0 auth_method 0 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe533fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe533fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe533fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:38.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe5337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.330+0000 7fe538b6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 10 monclient(hunting): handle_auth_done global_id 4933 payload 824 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 -- 10.20.193.18:0/1646567609 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe53415a5e0 msgr2=0x7fe5340bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe53415a5e0 0x7fe5340bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 -- 10.20.193.18:0/1646567609 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 msgr2=0x7fe5340cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 0x7fe5340cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 10 monclient: _finish_hunting 0 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 monclient: found mon.noname-b 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:38.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.331+0000 7fe5337fe640 1 -- 10.20.193.18:0/1646567609 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341f7dd0 con 0x7fe5340c99a0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.332+0000 7fe5337fe640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe5340bc4e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe524036000 tx=0x7fe5240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.332+0000 7fe530ff9640 10 client.?.objecter ms_handle_connect 0x7fe5340c99a0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.332+0000 7fe530ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524046020 con 0x7fe5340c99a0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 10 monclient: dump: 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.348 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 10 monclient: _finish_auth 0 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe530ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 5 monclient: authenticate success, global_id 4933 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 10 monclient: _renew_subs 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341f6f70 con 0x7fe5340c99a0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 10 client.4933.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 10 monclient: _renew_subs 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341f7530 con 0x7fe5340c99a0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.333+0000 7fe53a5f7640 1 librados: init done 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52402fa20 con 0x7fe5340c99a0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52403c040 con 0x7fe5340c99a0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe530ff9640 10 monclient: dump: 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:38.350 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe5327fc640 4 set_mon_vals no callback set 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe53a5f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe53a5f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe53a5f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.334+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe534150860 con 0x7fe5340c99a0 2026-02-24T18:38:38.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.345+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe524042850 con 0x7fe5340c99a0 2026-02-24T18:38:38.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.345+0000 7fe530ff9640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 0x7fe50407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:38.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.346+0000 7fe533fff640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 0x7fe50407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:38.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.346+0000 7fe533fff640 10 monclient: get_auth_request con 0x7fe504078e30 auth_method 0 2026-02-24T18:38:38.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.346+0000 7fe530ff9640 10 client.4933.objecter ms_handle_connect 0x7fe504078e30 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.346+0000 7fe533fff640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 0x7fe50407b2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe520002770 tx=0x7fe520009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.347+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fe5240bc140 con 0x7fe5340c99a0 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.347+0000 7fe530ff9640 10 client.4933.objecter ms_dispatch2 0x7fe534000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.347+0000 7fe530ff9640 3 client.4933.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.347+0000 7fe530ff9640 3 client.4933.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:38.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.347+0000 7fe530ff9640 20 client.4933.objecter dump_active .. 0 homeless 2026-02-24T18:38:38.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.447+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe524050020 con 0x7fe5340c99a0 2026-02-24T18:38:38.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.447+0000 7fe530ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:38.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.447+0000 7fe530ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:38.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.551+0000 7fe53a5f7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:38.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.551+0000 7fe53a5f7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:38.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.551+0000 7fe53a5f7640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:38.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.551+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe53414f2d0 con 0x7fe5340c99a0 2026-02-24T18:38:38.555 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.554+0000 7fe530ff9640 1 -- 10.20.193.18:0/1646567609 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fe524086a20 con 0x7fe5340c99a0 2026-02-24T18:38:38.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.554+0000 7fe530ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:38.556 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.554+0000 7fe530ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:38.556 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:38.556 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:38.558 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 10 librados: watch_flush enter 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 10 librados: watch_flush exit 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 20 client.4933.objecter shutdown clearing up homeless session... 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 10 client.4933.objecter successfully canceled tick 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 msgr2=0x7fe50407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 0x7fe50407b2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe520002770 tx=0x7fe520009040 comp rx=0 tx=0).stop 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 10 monclient: shutdown 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 msgr2=0x7fe5340bc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe5340bc4e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe524036000 tx=0x7fe5240047c0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 shutdown_connections 2026-02-24T18:38:38.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe504078e30 0x7fe50407b2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe5340c99a0 0x7fe5340bc4e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe53415a5e0 0x7fe5340bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 --2- 10.20.193.18:0/1646567609 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe53406ff20 0x7fe5340cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 >> 10.20.193.18:0/1646567609 conn(0x7fe5340cd780 msgr2=0x7fe53406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 shutdown_connections 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 -- 10.20.193.18:0/1646567609 wait complete. 2026-02-24T18:38:38.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:38.556+0000 7fe53a5f7640 1 librados: shutdown 2026-02-24T18:38:39.571 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:17.516 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.572 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:17.518 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.583 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:17.529 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.598 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:17.544 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.600 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:17.546 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.603 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:17.511 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.603 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:17.511 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.608 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:17.553 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:39.611 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:17.519 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:41.568 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:41.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 10 monclient: get_monmap_and_config 2026-02-24T18:38:41.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 10 monclient: build_initial_monmap 2026-02-24T18:38:41.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 10 monclient: monmap: 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:41.638261+0000 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:41.638261+0000 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:41.640 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:41.641 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 1 Processor -- start 2026-02-24T18:38:41.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 1 -- start start 2026-02-24T18:38:41.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.637+0000 7f3a5edd3640 10 monclient: init 2026-02-24T18:38:41.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:41.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient: _add_conns ranks=[1,3,0,5,2,4] 2026-02-24T18:38:41.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3a5806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:41.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3a580c99a0 0x7f3a58070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-a con 0x7f3a580c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3a5815a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a580c7e90 con 0x7f3a5815a5e0 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a580c8170 con 0x7f3a5806ff20 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3a58159a40 con 0x7f3a580c99a0 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5edd3640 10 monclient(hunting): authenticate will time out at 1665.319404s 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a4ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3a580c99a0 0x7f3a58070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5cb48640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f3a5806ff20 0x7f3a58159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:33202/0 (socket says 10.20.193.18:33202) 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.638+0000 7f3a5cb48640 1 -- 10.20.193.18:0/1948699799 learned_addr learned my addr 10.20.193.18:0/1948699799 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:41.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): get_auth_request con 0x7f3a580c99a0 auth_method 0 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.639+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 1 -- 10.20.193.18:0/1948699799 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262014741 0 0) 0x7f3a580c8170 con 0x7f3a5806ff20 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 10 monclient(hunting): my global_id is 4936 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4f7fe640 1 -- 10.20.193.18:0/1948699799 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a44003620 con 0x7f3a5806ff20 2026-02-24T18:38:41.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_done global_id 4953 payload 274 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 1 -- 10.20.193.18:0/1948699799 >> v1:10.20.193.32:6789/0 conn(0x7f3a5806ff20 legacy=0x7f3a58159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 1 -- 10.20.193.18:0/1948699799 >> v1:10.20.193.18:6789/0 conn(0x7f3a5815a5e0 legacy=0x7f3a5814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 1 monclient: found mon.noname-a 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.640+0000 7f3a4ffff640 1 -- 10.20.193.18:0/1948699799 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a580c8450 con 0x7f3a580c99a0 2026-02-24T18:38:41.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.642+0000 7f3a4ffff640 1 --2- 10.20.193.18:0/1948699799 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a580c99a0 0x7f3a58070300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a400098d0 tx=0x7f3a4002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b258751fd183ff61 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:41.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 1 -- 10.20.193.18:0/1948699799 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4003c070 con 0x7f3a580c99a0 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: dump: 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:41.652 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 1 -- 10.20.193.18:0/1948699799 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4002fba0 con 0x7f3a580c99a0 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 1 -- 10.20.193.18:0/1948699799 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40037440 con 0x7f3a580c99a0 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a4f7fe640 10 monclient: dump: 2026-02-24T18:38:41.653 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a5edd3640 5 monclient: authenticate success, global_id 4953 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a5edd3640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:41.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.644+0000 7f3a5edd3640 4 set_mon_vals no callback set 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 10 monclient: shutdown 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 -- 10.20.193.18:0/1948699799 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a580c99a0 msgr2=0x7f3a58070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/1948699799 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a580c99a0 0x7f3a58070300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a400098d0 tx=0x7f3a4002eed0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 -- 10.20.193.18:0/1948699799 shutdown_connections 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/1948699799 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a580c99a0 0x7f3a58070300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 -- 10.20.193.18:0/1948699799 >> 10.20.193.18:0/1948699799 conn(0x7f3a580cd780 msgr2=0x7f3a580cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 -- 10.20.193.18:0/1948699799 shutdown_connections 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 1 -- 10.20.193.18:0/1948699799 wait complete. 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 10 monclient: build_initial_monmap 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.645+0000 7f3a5edd3640 10 monclient: monmap: 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:41.655 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 librados: starting msgr at 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 librados: starting objecter 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 Processor -- start 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 -- start start 2026-02-24T18:38:41.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 librados: setting wanted keys 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 librados: calling monclient init 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient: init 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-a con 0x7f3a5806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a580c99a0 0x7f3a580c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-b con 0x7f3a580c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a5815a5e0 0x7f3a580bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): picked mon.noname-c con 0x7f3a5815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:41.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58150520 con 0x7f3a5806ff20 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580c8a10 con 0x7f3a580c99a0 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581506a0 con 0x7f3a5815a5e0 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5edd3640 10 monclient(hunting): authenticate will time out at 1665.327626s 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.646+0000 7f3a5cb48640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a580c99a0 0x7f3a580c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:36844/0 (socket says 10.20.193.18:36844) 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 1 -- 10.20.193.18:0/371800173 learned_addr learned my addr 10.20.193.18:0/371800173 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): get_auth_request con 0x7f3a5806ff20 auth_method 0 2026-02-24T18:38:41.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a5815a5e0 0x7f3a580bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5cb48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): get_auth_request con 0x7f3a580c99a0 auth_method 0 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): get_auth_request con 0x7f3a5815a5e0 auth_method 0 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:41.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.647+0000 7f3a5d349640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 10 monclient(hunting): handle_auth_done global_id 4959 payload 824 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 -- 10.20.193.18:0/371800173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a5815a5e0 msgr2=0x7f3a580bb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a5815a5e0 0x7f3a580bb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 -- 10.20.193.18:0/371800173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a580c99a0 msgr2=0x7f3a580c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a580c99a0 0x7f3a580c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 10 monclient: _finish_hunting 0 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 monclient: found mon.noname-a 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.648+0000 7f3a5cb48640 1 -- 10.20.193.18:0/371800173 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a580c0a30 con 0x7f3a5806ff20 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.649+0000 7f3a5cb48640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c700 tx=0x7f3a4800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.649+0000 7f3a4d7fa640 10 client.?.objecter ms_handle_connect 0x7f3a5806ff20 2026-02-24T18:38:41.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.649+0000 7f3a4d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48017070 con 0x7f3a5806ff20 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: dump: 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:41.663 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48005bf0 con 0x7f3a5806ff20 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48007070 con 0x7f3a5806ff20 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:41.664 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4effd640 4 set_mon_vals no callback set 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a4d7fa640 10 monclient: dump: 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:41.665 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 5 monclient: authenticate success, global_id 4959 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 10 monclient: _renew_subs 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a580be550 con 0x7f3a5806ff20 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 10 client.4959.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 10 monclient: _renew_subs 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a580beb10 con 0x7f3a5806ff20 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.651+0000 7f3a5edd3640 1 librados: init done 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.652+0000 7f3a5edd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.652+0000 7f3a5edd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:41.666 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.652+0000 7f3a5edd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.652+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a58070300 con 0x7f3a5806ff20 2026-02-24T18:38:41.667 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.663+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3a48007210 con 0x7f3a5806ff20 2026-02-24T18:38:41.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.664+0000 7f3a4d7fa640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 0x7f3a2807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:41.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.664+0000 7f3a4ffff640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 0x7f3a2807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:41.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.664+0000 7f3a4ffff640 10 monclient: get_auth_request con 0x7f3a28078e80 auth_method 0 2026-02-24T18:38:41.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.665+0000 7f3a4d7fa640 10 client.4959.objecter ms_handle_connect 0x7f3a28078e80 2026-02-24T18:38:41.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.665+0000 7f3a4ffff640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 0x7f3a2807b340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a40039000 tx=0x7f3a40033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:41.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.666+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f3a4805d470 con 0x7f3a5806ff20 2026-02-24T18:38:41.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.666+0000 7f3a4d7fa640 10 client.4959.objecter ms_dispatch2 0x7f3a58000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:41.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.666+0000 7f3a4d7fa640 3 client.4959.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:41.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.666+0000 7f3a4d7fa640 3 client.4959.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:41.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.666+0000 7f3a4d7fa640 20 client.4959.objecter dump_active .. 0 homeless 2026-02-24T18:38:41.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.767+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3a48021020 con 0x7f3a5806ff20 2026-02-24T18:38:41.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.767+0000 7f3a4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:41.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.767+0000 7f3a4d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:41.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.874+0000 7f3a5edd3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:41.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.874+0000 7f3a5edd3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:41.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.874+0000 7f3a5edd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:41.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.874+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3a5814f380 con 0x7f3a5806ff20 2026-02-24T18:38:41.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.876+0000 7f3a4d7fa640 1 -- 10.20.193.18:0/371800173 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3a48061430 con 0x7f3a5806ff20 2026-02-24T18:38:41.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.876+0000 7f3a4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:41.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.876+0000 7f3a4d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:41.878 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:41.878 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:41.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 10 librados: watch_flush enter 2026-02-24T18:38:41.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 10 librados: watch_flush exit 2026-02-24T18:38:41.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 20 client.4959.objecter shutdown clearing up homeless session... 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 10 client.4959.objecter successfully canceled tick 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 msgr2=0x7f3a2807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 0x7f3a2807b340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a40039000 tx=0x7f3a40033000 comp rx=0 tx=0).stop 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 10 monclient: shutdown 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 msgr2=0x7f3a580c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c700 tx=0x7f3a4800cbd0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 shutdown_connections 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3a28078e80 0x7f3a2807b340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3a580c99a0 0x7f3a580c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3a5815a5e0 0x7f3a580bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 --2- 10.20.193.18:0/371800173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3a5806ff20 0x7f3a580c04f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:41.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 >> 10.20.193.18:0/371800173 conn(0x7f3a580cd780 msgr2=0x7f3a5814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:41.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 shutdown_connections 2026-02-24T18:38:41.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 -- 10.20.193.18:0/371800173 wait complete. 2026-02-24T18:38:41.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:41.878+0000 7f3a5edd3640 1 librados: shutdown 2026-02-24T18:38:44.891 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:44.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 10 monclient: get_monmap_and_config 2026-02-24T18:38:44.962 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 10 monclient: build_initial_monmap 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 10 monclient: monmap: 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:44.961365+0000 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:44.961365+0000 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:44.963 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 1 Processor -- start 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 1 -- start start 2026-02-24T18:38:44.964 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.960+0000 7ff59b153640 10 monclient: init 2026-02-24T18:38:44.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:44.966 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient: _add_conns ranks=[1,0,4,5,3,2] 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff59406ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5940c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff59415a5e0 0x7ff594070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-b con 0x7ff59415a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5940c7e90 con 0x7ff5940c99a0 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5940c8170 con 0x7ff59406ff20 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff594159a40 con 0x7ff59415a5e0 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff59b153640 10 monclient(hunting): authenticate will time out at 1668.642545s 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff58bfff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7ff5940c99a0 0x7ff59414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:47046/0 (socket says 10.20.193.18:47046) 2026-02-24T18:38:44.967 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.961+0000 7ff58bfff640 1 -- 10.20.193.18:0/66797496 learned_addr learned my addr 10.20.193.18:0/66797496 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 1 --2- 10.20.193.18:0/66797496 >> v2:10.20.193.32:3300/0 conn(0x7ff59415a5e0 0x7ff594070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): get_auth_request con 0x7ff59415a5e0 auth_method 0 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.962+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 1 -- 10.20.193.18:0/66797496 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170658083 0 0) 0x7ff5940c7e90 con 0x7ff5940c99a0 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 10 monclient(hunting): _negotiate 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:44.968 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 10 monclient(hunting): my global_id is 4962 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff58b7fe640 1 -- 10.20.193.18:0/66797496 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff580003620 con 0x7ff5940c99a0 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 10 monclient(hunting): handle_auth_done global_id 4942 payload 274 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 1 -- 10.20.193.18:0/66797496 >> v1:10.20.193.32:6789/0 conn(0x7ff59406ff20 legacy=0x7ff594159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 1 -- 10.20.193.18:0/66797496 >> v1:10.20.193.18:6789/0 conn(0x7ff5940c99a0 legacy=0x7ff59414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 10 monclient: _finish_hunting 0 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 1 monclient: found mon.noname-b 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.963+0000 7ff5996c9640 1 -- 10.20.193.18:0/66797496 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940c8450 con 0x7ff59415a5e0 2026-02-24T18:38:44.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.965+0000 7ff5996c9640 1 --2- 10.20.193.18:0/66797496 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff59415a5e0 0x7ff594070300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff590009dc0 tx=0x7ff59002f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cab3f1caacddd82 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:44.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 1 -- 10.20.193.18:0/66797496 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59003c070 con 0x7ff59415a5e0 2026-02-24T18:38:44.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:44.974 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: dump: 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:44.975 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 1 -- 10.20.193.18:0/66797496 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff590037440 con 0x7ff59415a5e0 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 5 monclient: authenticate success, global_id 4942 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 1 -- 10.20.193.18:0/66797496 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590037810 con 0x7ff59415a5e0 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff58b7fe640 10 monclient: dump: 2026-02-24T18:38:44.976 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:44.977 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 4 set_mon_vals no callback set 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 10 monclient: shutdown 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 -- 10.20.193.18:0/66797496 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff59415a5e0 msgr2=0x7ff594070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 --2- 10.20.193.18:0/66797496 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff59415a5e0 0x7ff594070300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff590009dc0 tx=0x7ff59002f230 comp rx=0 tx=0).stop 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 -- 10.20.193.18:0/66797496 shutdown_connections 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 --2- 10.20.193.18:0/66797496 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff59415a5e0 0x7ff594070300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:44.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 -- 10.20.193.18:0/66797496 >> 10.20.193.18:0/66797496 conn(0x7ff5940cd780 msgr2=0x7ff5940cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 -- 10.20.193.18:0/66797496 shutdown_connections 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.967+0000 7ff59b153640 1 -- 10.20.193.18:0/66797496 wait complete. 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient: build_initial_monmap 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient: monmap: 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.979 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 librados: starting msgr at 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 librados: starting objecter 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 Processor -- start 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 -- start start 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 librados: setting wanted keys 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 librados: calling monclient init 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient: init 2026-02-24T18:38:44.980 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 0x7ff594148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-a con 0x7ff59406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 0x7ff5941f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-b con 0x7ff5940c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff59415a5e0 0x7ff5941f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): picked mon.noname-c con 0x7ff59415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff594150520 con 0x7ff59406ff20 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940c8a10 con 0x7ff5940c99a0 2026-02-24T18:38:44.981 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941506a0 con 0x7ff59415a5e0 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.968+0000 7ff59b153640 10 monclient(hunting): authenticate will time out at 1668.649999s 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 0x7ff594148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 0x7ff5941f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 0x7ff594148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:36860/0 (socket says 10.20.193.18:36860) 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 1 -- 10.20.193.18:0/1950831989 learned_addr learned my addr 10.20.193.18:0/1950831989 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): get_auth_request con 0x7ff59406ff20 auth_method 0 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:44.982 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): get_auth_request con 0x7ff5940c99a0 auth_method 0 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff5996c9640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff59415a5e0 0x7ff5941f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff598ec8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff5996c9640 10 monclient(hunting): get_auth_request con 0x7ff59415a5e0 auth_method 0 2026-02-24T18:38:44.983 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff5996c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff5996c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.969+0000 7ff5996c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff5996c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 10 monclient(hunting): handle_auth_done global_id 4945 payload 824 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 1 -- 10.20.193.18:0/1950831989 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff59415a5e0 msgr2=0x7ff5941f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff59415a5e0 0x7ff5941f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 1 -- 10.20.193.18:0/1950831989 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 msgr2=0x7ff594148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 0x7ff594148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 1 monclient: found mon.noname-b 2026-02-24T18:38:44.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.970+0000 7ff58bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:44.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.971+0000 7ff58bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:44.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.971+0000 7ff58bfff640 1 -- 10.20.193.18:0/1950831989 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5941479b0 con 0x7ff5940c99a0 2026-02-24T18:38:44.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.971+0000 7ff58bfff640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 0x7ff5941f2360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c500 tx=0x7ff57c00c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:44.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.971+0000 7ff5897fa640 10 client.?.objecter ms_handle_connect 0x7ff5940c99a0 2026-02-24T18:38:44.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.971+0000 7ff5897fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:44.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c017070 con 0x7ff5940c99a0 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: dump: 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:44.988 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: _finish_auth 0 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: _check_auth_tickets 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c005de0 con 0x7ff5940c99a0 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c007270 con 0x7ff5940c99a0 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff5897fa640 10 monclient: dump: 2026-02-24T18:38:44.989 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:44.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff58affd640 4 set_mon_vals no callback set 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 5 monclient: authenticate success, global_id 4945 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 10 monclient: _renew_subs 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff594147c90 con 0x7ff5940c99a0 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 10 client.4945.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 10 monclient: _renew_subs 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff594148250 con 0x7ff5940c99a0 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.973+0000 7ff59b153640 1 librados: init done 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.974+0000 7ff59b153640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.974+0000 7ff59b153640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.974+0000 7ff59b153640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:44.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.974+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff594070300 con 0x7ff5940c99a0 2026-02-24T18:38:44.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.985+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff57c007410 con 0x7ff5940c99a0 2026-02-24T18:38:44.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 0x7ff56407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:44.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff598ec8640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 0x7ff56407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:44.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff598ec8640 10 monclient: get_auth_request con 0x7ff564078e00 auth_method 0 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7ff57c007730 con 0x7ff5940c99a0 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 10 client.4945.objecter ms_dispatch2 0x7ff594000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 3 client.4945.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 3 client.4945.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff598ec8640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 0x7ff56407b2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff584004660 tx=0x7ff584009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 20 client.4945.objecter dump_active .. 0 homeless 2026-02-24T18:38:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:44.986+0000 7ff5897fa640 10 client.4945.objecter ms_handle_connect 0x7ff564078e00 2026-02-24T18:38:45.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.083+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff57c061510 con 0x7ff5940c99a0 2026-02-24T18:38:45.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.083+0000 7ff5897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:45.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.083+0000 7ff5897fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:45.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.184+0000 7ff59b153640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:45.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.184+0000 7ff59b153640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:45.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.184+0000 7ff59b153640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:45.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.184+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff5940c4c50 con 0x7ff5940c99a0 2026-02-24T18:38:45.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.186+0000 7ff5897fa640 1 -- 10.20.193.18:0/1950831989 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff57c065810 con 0x7ff5940c99a0 2026-02-24T18:38:45.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.186+0000 7ff5897fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:45.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.186+0000 7ff5897fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:45.188 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:45.188 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 10 librados: watch_flush enter 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 10 librados: watch_flush exit 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 20 client.4945.objecter shutdown clearing up homeless session... 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 10 client.4945.objecter successfully canceled tick 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 msgr2=0x7ff56407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:45.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 0x7ff56407b2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff584004660 tx=0x7ff584009290 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 10 monclient: shutdown 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 msgr2=0x7ff5941f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 0x7ff5941f2360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c500 tx=0x7ff57c00c9d0 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 shutdown_connections 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff564078e00 0x7ff56407b2c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff5940c99a0 0x7ff5941f2360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff59415a5e0 0x7ff5941f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 --2- 10.20.193.18:0/1950831989 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff59406ff20 0x7ff594148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 >> 10.20.193.18:0/1950831989 conn(0x7ff5940cd780 msgr2=0x7ff59414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 shutdown_connections 2026-02-24T18:38:45.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 -- 10.20.193.18:0/1950831989 wait complete. 2026-02-24T18:38:45.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:45.187+0000 7ff59b153640 1 librados: shutdown 2026-02-24T18:38:48.198 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:48.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.263+0000 7f00325e4640 10 monclient: get_monmap_and_config 2026-02-24T18:38:48.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.263+0000 7f00325e4640 10 monclient: build_initial_monmap 2026-02-24T18:38:48.270 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.263+0000 7f00325e4640 10 monclient: monmap: 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:48.265589+0000 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:48.265589+0000 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:48.271 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 Processor -- start 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 -- start start 2026-02-24T18:38:48.272 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient: init 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient: _add_conns ranks=[4,2,0,3,1,5] 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f002c1527d0 0x7f002c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-b con 0x7f002c1527d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f002c152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f002c0c6190 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002c06e300 con 0x7f002c0c6190 2026-02-24T18:38:48.273 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002c06e5e0 con 0x7f002c152bb0 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f002c06f1f0 con 0x7f002c1527d0 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.264+0000 7f00325e4640 10 monclient(hunting): authenticate will time out at 1671.946743s 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f0030b5a640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f002c0c6190 0x7f002c0c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:47062/0 (socket says 10.20.193.18:47062) 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f0030b5a640 1 -- 10.20.193.18:0/1032129397 learned_addr learned my addr 10.20.193.18:0/1032129397 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 1 --2- 10.20.193.18:0/1032129397 >> v2:10.20.193.32:3300/0 conn(0x7f002c1527d0 0x7f002c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:48.274 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): get_auth_request con 0x7f002c1527d0 auth_method 0 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.265+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1987188246 0 0) 0x7f002c06e300 con 0x7f002c0c6190 2026-02-24T18:38:48.275 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _negotiate 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): my global_id is 4968 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0010003620 con 0x7f002c0c6190 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529512268 0 0) 0x7f002c06e5e0 con 0x7f002c152bb0 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _negotiate 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.276 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 10 monclient(hunting): my global_id is 4913 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.266+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002c06e300 con 0x7f002c152bb0 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 10 monclient(hunting): handle_auth_done global_id 4948 payload 274 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 1 -- 10.20.193.18:0/1032129397 >> v1:10.20.193.110:6789/0 conn(0x7f002c152bb0 legacy=0x7f002c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 1 -- 10.20.193.18:0/1032129397 >> v1:10.20.193.18:6789/0 conn(0x7f002c0c6190 legacy=0x7f002c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 1 monclient: found mon.noname-b 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.277 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.267+0000 7f002bfff640 1 -- 10.20.193.18:0/1032129397 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c06e8c0 con 0x7f002c1527d0 2026-02-24T18:38:48.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.268+0000 7f002bfff640 1 --2- 10.20.193.18:0/1032129397 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c1527d0 0x7f002c06da60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f002c06f370 tx=0x7f001802ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32498844a9fd282e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:48.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001803c070 con 0x7f002c1527d0 2026-02-24T18:38:48.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: dump: 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:48.285 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: _finish_auth 0 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001802fb70 con 0x7f002c1527d0 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 1 -- 10.20.193.18:0/1032129397 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018037440 con 0x7f002c1527d0 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f002affd640 10 monclient: dump: 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:48.286 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 5 monclient: authenticate success, global_id 4948 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 4 set_mon_vals no callback set 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 10 monclient: shutdown 2026-02-24T18:38:48.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 -- 10.20.193.18:0/1032129397 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c1527d0 msgr2=0x7f002c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 --2- 10.20.193.18:0/1032129397 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c1527d0 0x7f002c06da60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f002c06f370 tx=0x7f001802ee60 comp rx=0 tx=0).stop 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 -- 10.20.193.18:0/1032129397 shutdown_connections 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 --2- 10.20.193.18:0/1032129397 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c1527d0 0x7f002c06da60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 -- 10.20.193.18:0/1032129397 >> 10.20.193.18:0/1032129397 conn(0x7f002c0cd780 msgr2=0x7f002c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 -- 10.20.193.18:0/1032129397 shutdown_connections 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.270+0000 7f00325e4640 1 -- 10.20.193.18:0/1032129397 wait complete. 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient: build_initial_monmap 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient: monmap: 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:48.288 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 librados: starting msgr at 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 librados: starting objecter 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 Processor -- start 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 -- start start 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 librados: setting wanted keys 2026-02-24T18:38:48.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 librados: calling monclient init 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient: init 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-b con 0x7f002c0c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 0x7f002c1ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-a con 0x7f002c1527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f002c152bb0 0x7f002c0734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): picked mon.noname-c con 0x7f002c152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f002c06dfa0 con 0x7f002c1527d0 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:48.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f002c06ee80 con 0x7f002c0c6190 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f002c0c9570 con 0x7f002c152bb0 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.271+0000 7f00325e4640 10 monclient(hunting): authenticate will time out at 1671.953810s 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 0x7f002c1ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:55396/0 (socket says 10.20.193.18:55396) 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 0x7f002c1ed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:36874/0 (socket says 10.20.193.18:36874) 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 1 -- 10.20.193.18:0/2267613289 learned_addr learned my addr 10.20.193.18:0/2267613289 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): get_auth_request con 0x7f002c0c6190 auth_method 0 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:48.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): get_auth_request con 0x7f002c1527d0 auth_method 0 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f002c152bb0 0x7f002c0734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): get_auth_request con 0x7f002c152bb0 auth_method 0 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:48.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f002b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.272+0000 7f0030b5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 10 monclient(hunting): handle_auth_done global_id 4951 payload 824 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 -- 10.20.193.18:0/2267613289 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f002c152bb0 msgr2=0x7f002c0734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f002c152bb0 0x7f002c0734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 -- 10.20.193.18:0/2267613289 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 msgr2=0x7f002c1ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 0x7f002c1ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:38:48.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 monclient: found mon.noname-b 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.273+0000 7f002bfff640 1 -- 10.20.193.18:0/2267613289 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c0c2270 con 0x7f002c0c6190 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.275+0000 7f002bfff640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f001803a040 tx=0x7f00180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.275+0000 7f0028ff9640 10 client.?.objecter ms_handle_connect 0x7f002c0c6190 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.275+0000 7f0028ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018046020 con 0x7f002c0c6190 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: dump: 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:48.294 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: _finish_auth 0 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0018042450 con 0x7f002c0c6190 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:48.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001803c040 con 0x7f002c0c6190 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f0028ff9640 10 monclient: dump: 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:48.296 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f002a7fc640 4 set_mon_vals no callback set 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 5 monclient: authenticate success, global_id 4951 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 10 monclient: _renew_subs 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002c0c2550 con 0x7f002c0c6190 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 10 client.4951.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 10 monclient: _renew_subs 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002c0c2b10 con 0x7f002c0c6190 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.276+0000 7f00325e4640 1 librados: init done 2026-02-24T18:38:48.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.277+0000 7f00325e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:48.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.277+0000 7f00325e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:48.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.277+0000 7f00325e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.277+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002c06da60 con 0x7f002c0c6190 2026-02-24T18:38:48.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.288+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f001804a3b0 con 0x7f002c0c6190 2026-02-24T18:38:48.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.289+0000 7f0028ff9640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 0x7efffc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:48.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.289+0000 7f002b7fe640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 0x7efffc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:48.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.289+0000 7f002b7fe640 10 monclient: get_auth_request con 0x7efffc078e00 auth_method 0 2026-02-24T18:38:48.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.289+0000 7f002b7fe640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 0x7efffc07b2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f001c002930 tx=0x7f001c003d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.289+0000 7f0028ff9640 10 client.4951.objecter ms_handle_connect 0x7efffc078e00 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.291+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f0018082a50 con 0x7f002c0c6190 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.291+0000 7f0028ff9640 10 client.4951.objecter ms_dispatch2 0x7f002c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.291+0000 7f0028ff9640 3 client.4951.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.291+0000 7f0028ff9640 3 client.4951.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:48.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.291+0000 7f0028ff9640 20 client.4951.objecter dump_active .. 0 homeless 2026-02-24T18:38:48.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.393+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0018050020 con 0x7f002c0c6190 2026-02-24T18:38:48.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.393+0000 7f0028ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:48.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.393+0000 7f0028ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:48.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.495+0000 7f00325e4640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:48.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.495+0000 7f00325e4640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:48.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.495+0000 7f00325e4640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:48.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.495+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f002c0c8220 con 0x7f002c0c6190 2026-02-24T18:38:48.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.498+0000 7f0028ff9640 1 -- 10.20.193.18:0/2267613289 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0018086a10 con 0x7f002c0c6190 2026-02-24T18:38:48.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.498+0000 7f0028ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:48.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.498+0000 7f0028ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:48.500 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:48.500 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:48.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 10 librados: watch_flush enter 2026-02-24T18:38:48.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 10 librados: watch_flush exit 2026-02-24T18:38:48.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 20 client.4951.objecter shutdown clearing up homeless session... 2026-02-24T18:38:48.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 10 client.4951.objecter successfully canceled tick 2026-02-24T18:38:48.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 msgr2=0x7efffc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 0x7efffc07b2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f001c002930 tx=0x7f001c003d10 comp rx=0 tx=0).stop 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 10 monclient: shutdown 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 msgr2=0x7f002c0c38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f001803a040 tx=0x7f00180047c0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 shutdown_connections 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efffc078e00 0x7efffc07b2c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f002c152bb0 0x7f002c0734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f002c1527d0 0x7f002c1ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 --2- 10.20.193.18:0/2267613289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f002c0c6190 0x7f002c0c38d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:48.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 >> 10.20.193.18:0/2267613289 conn(0x7f002c0cd780 msgr2=0x7f002c0c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:48.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 shutdown_connections 2026-02-24T18:38:48.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 -- 10.20.193.18:0/2267613289 wait complete. 2026-02-24T18:38:48.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:48.499+0000 7f00325e4640 1 librados: shutdown 2026-02-24T18:38:51.512 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:51.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.582+0000 7f2aa4d78640 10 monclient: get_monmap_and_config 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.582+0000 7f2aa4d78640 10 monclient: build_initial_monmap 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.582+0000 7f2aa4d78640 10 monclient: monmap: 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:51.583527+0000 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:51.583527+0000 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:51.586 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 Processor -- start 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 -- start start 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient: init 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:51.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient: _add_conns ranks=[1,0,3,4,2,5] 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2aa006ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2aa00c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2aa015a5e0 0x7f2aa0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-a con 0x7f2aa015a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa00c7e90 con 0x7f2aa00c99a0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa00c8170 con 0x7f2aa006ff20 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa0159a40 con 0x7f2aa015a5e0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.583+0000 7f2aa4d78640 10 monclient(hunting): authenticate will time out at 1675.264638s 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f2aa015a5e0 0x7f2aa0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:51.588 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9dd74640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f2aa00c99a0 0x7f2aa014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58598/0 (socket says 10.20.193.18:58598) 2026-02-24T18:38:51.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9dd74640 1 -- 10.20.193.18:0/2815494354 learned_addr learned my addr 10.20.193.18:0/2815494354 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:51.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): get_auth_request con 0x7f2aa015a5e0 auth_method 0 2026-02-24T18:38:51.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:51.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:51.589 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:51.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:51.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:51.590 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.584+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 1 -- 10.20.193.18:0/2815494354 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888693121 0 0) 0x7f2aa00c8170 con 0x7f2aa006ff20 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 10 monclient(hunting): _negotiate 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 10 monclient(hunting): my global_id is 4954 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9d573640 1 -- 10.20.193.18:0/2815494354 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a8c003620 con 0x7f2aa006ff20 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_done global_id 4974 payload 274 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 1 -- 10.20.193.18:0/2815494354 >> v1:10.20.193.32:6789/0 conn(0x7f2aa006ff20 legacy=0x7f2aa0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 1 -- 10.20.193.18:0/2815494354 >> v1:10.20.193.18:6789/0 conn(0x7f2aa00c99a0 legacy=0x7f2aa014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 10 monclient: _finish_hunting 0 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 1 monclient: found mon.noname-a 2026-02-24T18:38:51.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:51.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.586+0000 7f2a9ed76640 1 -- 10.20.193.18:0/2815494354 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa00c8450 con 0x7f2aa015a5e0 2026-02-24T18:38:51.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.588+0000 7f2a9ed76640 1 --2- 10.20.193.18:0/2815494354 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa015a5e0 0x7f2aa0070300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009dc0 tx=0x7f2a9002f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f958befe801999 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:51.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 1 -- 10.20.193.18:0/2815494354 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9003c070 con 0x7f2aa015a5e0 2026-02-24T18:38:51.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: dump: 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:51.597 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: _finish_auth 0 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: _check_auth_tickets 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 1 -- 10.20.193.18:0/2815494354 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a90037440 con 0x7f2aa015a5e0 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 1 -- 10.20.193.18:0/2815494354 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90037810 con 0x7f2aa015a5e0 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 5 monclient: authenticate success, global_id 4974 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 4 set_mon_vals no callback set 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:51.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2a9d573640 10 monclient: dump: 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 10 monclient: shutdown 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 -- 10.20.193.18:0/2815494354 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa015a5e0 msgr2=0x7f2aa0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/2815494354 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa015a5e0 0x7f2aa0070300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009dc0 tx=0x7f2a9002f280 comp rx=0 tx=0).stop 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 -- 10.20.193.18:0/2815494354 shutdown_connections 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/2815494354 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa015a5e0 0x7f2aa0070300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 -- 10.20.193.18:0/2815494354 >> 10.20.193.18:0/2815494354 conn(0x7f2aa00cd780 msgr2=0x7f2aa00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 -- 10.20.193.18:0/2815494354 shutdown_connections 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.590+0000 7f2aa4d78640 1 -- 10.20.193.18:0/2815494354 wait complete. 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient: build_initial_monmap 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient: monmap: 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:51.600 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 librados: starting msgr at 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 librados: starting objecter 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 Processor -- start 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 -- start start 2026-02-24T18:38:51.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 librados: setting wanted keys 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 librados: calling monclient init 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient: init 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2aa006ff20 0x7f2aa00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-b con 0x7f2aa006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-a con 0x7f2aa00c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2aa015a5e0 0x7f2aa01f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): picked mon.noname-c con 0x7f2aa015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0150520 con 0x7f2aa00c99a0 2026-02-24T18:38:51.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa00c8a10 con 0x7f2aa006ff20 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01506a0 con 0x7f2aa015a5e0 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.591+0000 7f2aa4d78640 10 monclient(hunting): authenticate will time out at 1675.272906s 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2aa006ff20 0x7f2aa00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55054/0 (socket says 10.20.193.18:55054) 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 1 -- 10.20.193.18:0/4206566143 learned_addr learned my addr 10.20.193.18:0/4206566143 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): get_auth_request con 0x7f2aa00c99a0 auth_method 0 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:51.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9ed76640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2aa015a5e0 0x7f2aa01f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): get_auth_request con 0x7f2aa006ff20 auth_method 0 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9ed76640 10 monclient(hunting): get_auth_request con 0x7f2aa015a5e0 auth_method 0 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9ed76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:51.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.592+0000 7f2a9e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 10 monclient(hunting): handle_auth_done global_id 4980 payload 824 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 -- 10.20.193.18:0/4206566143 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2aa015a5e0 msgr2=0x7f2aa01f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2aa015a5e0 0x7f2aa01f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 -- 10.20.193.18:0/4206566143 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2aa006ff20 msgr2=0x7f2aa00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2aa006ff20 0x7f2aa00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 10 monclient: _finish_hunting 0 2026-02-24T18:38:51.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 monclient: found mon.noname-a 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.593+0000 7f2a9dd74640 1 -- 10.20.193.18:0/4206566143 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa00bd2b0 con 0x7f2aa00c99a0 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.594+0000 7f2a9dd74640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c700 tx=0x7f2a8800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.594+0000 7f2a86ffd640 10 client.?.objecter ms_handle_connect 0x7f2aa00c99a0 2026-02-24T18:38:51.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.594+0000 7f2a86ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88017070 con 0x7f2aa00c99a0 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: dump: 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.609 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: _finish_auth 0 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a88005bb0 con 0x7f2aa00c99a0 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 5 monclient: authenticate success, global_id 4980 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 10 monclient: _renew_subs 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa01f4a70 con 0x7f2aa00c99a0 2026-02-24T18:38:51.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88007070 con 0x7f2aa00c99a0 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a9cd72640 4 set_mon_vals no callback set 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2a86ffd640 10 monclient: dump: 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:51.611 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 10 client.4980.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 10 monclient: _renew_subs 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01f5030 con 0x7f2aa00c99a0 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.596+0000 7f2aa4d78640 1 librados: init done 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.597+0000 7f2aa4d78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.597+0000 7f2aa4d78640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.597+0000 7f2aa4d78640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.597+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa0070300 con 0x7f2aa00c99a0 2026-02-24T18:38:51.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.608+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2a88007210 con 0x7f2aa00c99a0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.608+0000 7f2a86ffd640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 0x7f2a6c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.609+0000 7f2a9e575640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 0x7f2a6c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.609+0000 7f2a9e575640 10 monclient: get_auth_request con 0x7f2a6c078e50 auth_method 0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.609+0000 7f2a9e575640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 0x7f2a6c07b310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a94003fd0 tx=0x7f2a94007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.609+0000 7f2a86ffd640 10 client.4980.objecter ms_handle_connect 0x7f2a6c078e50 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.610+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f2a8805d4f0 con 0x7f2aa00c99a0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.610+0000 7f2a86ffd640 10 client.4980.objecter ms_dispatch2 0x7f2aa0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.610+0000 7f2a86ffd640 3 client.4980.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.610+0000 7f2a86ffd640 3 client.4980.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:51.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.610+0000 7f2a86ffd640 20 client.4980.objecter dump_active .. 0 homeless 2026-02-24T18:38:51.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.709+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2a88022020 con 0x7f2aa00c99a0 2026-02-24T18:38:51.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.709+0000 7f2a86ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:51.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.709+0000 7f2a86ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:51.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.809+0000 7f2aa4d78640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:51.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.809+0000 7f2aa4d78640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:51.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.809+0000 7f2aa4d78640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:38:51.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.809+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2aa00c4a10 con 0x7f2aa00c99a0 2026-02-24T18:38:51.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.815+0000 7f2a86ffd640 1 -- 10.20.193.18:0/4206566143 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f2a880614b0 con 0x7f2aa00c99a0 2026-02-24T18:38:51.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.815+0000 7f2a86ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:51.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.815+0000 7f2a86ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:51.817 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:51.817 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 10 librados: watch_flush enter 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 10 librados: watch_flush exit 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 20 client.4980.objecter shutdown clearing up homeless session... 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 10 client.4980.objecter successfully canceled tick 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 msgr2=0x7f2a6c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 0x7f2a6c07b310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a94003fd0 tx=0x7f2a94007dc0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 10 monclient: shutdown 2026-02-24T18:38:51.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 msgr2=0x7f2aa01f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c700 tx=0x7f2a8800cbd0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 shutdown_connections 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2a6c078e50 0x7f2a6c07b310 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2aa015a5e0 0x7f2aa01f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2aa00c99a0 0x7f2aa01f1570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 --2- 10.20.193.18:0/4206566143 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2aa006ff20 0x7f2aa00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 >> 10.20.193.18:0/4206566143 conn(0x7f2aa00cd780 msgr2=0x7f2aa014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 shutdown_connections 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 -- 10.20.193.18:0/4206566143 wait complete. 2026-02-24T18:38:51.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:51.816+0000 7f2aa4d78640 1 librados: shutdown 2026-02-24T18:38:54.828 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:54.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 10 monclient: get_monmap_and_config 2026-02-24T18:38:54.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 10 monclient: build_initial_monmap 2026-02-24T18:38:54.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 10 monclient: monmap: 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:54.898068+0000 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:54.898068+0000 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:54.899 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:54.900 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:54.900 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:54.900 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 1 Processor -- start 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 1 -- start start 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.897+0000 7faa9bb85640 10 monclient: init 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient: _add_conns ranks=[5,0,1,4,2,3] 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7faa941527d0 0x7faa9406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-c con 0x7faa941527d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faa94152bb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7faa940c6190 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9406e300 con 0x7faa94152bb0 2026-02-24T18:38:54.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9406e5e0 con 0x7faa940c6190 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7faa9406f1f0 con 0x7faa941527d0 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa9bb85640 10 monclient(hunting): authenticate will time out at 1678.579190s 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa990f9640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7faa94152bb0 0x7faa940c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:58602/0 (socket says 10.20.193.18:58602) 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa990f9640 1 -- 10.20.193.18:0/3696417538 learned_addr learned my addr 10.20.193.18:0/3696417538 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.898+0000 7faa998fa640 1 --2- 10.20.193.18:0/3696417538 >> v2:10.20.193.110:3300/0 conn(0x7faa941527d0 0x7faa9406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): get_auth_request con 0x7faa941527d0 auth_method 0 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013107854 0 0) 0x7faa9406e300 con 0x7faa94152bb0 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 10 monclient(hunting): _negotiate 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 10 monclient(hunting): my global_id is 4983 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.899+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa7c003620 con 0x7faa94152bb0 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976233233 0 0) 0x7faa9406e5e0 con 0x7faa940c6190 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 10 monclient(hunting): _negotiate 2026-02-24T18:38:54.905 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 10 monclient(hunting): my global_id is 4960 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa9406e300 con 0x7faa940c6190 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 10 monclient(hunting): handle_auth_done global_id 4922 payload 274 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 1 -- 10.20.193.18:0/3696417538 >> v1:10.20.193.32:6789/0 conn(0x7faa940c6190 legacy=0x7faa940c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 1 -- 10.20.193.18:0/3696417538 >> v1:10.20.193.18:6789/0 conn(0x7faa94152bb0 legacy=0x7faa940c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 10 monclient: _finish_hunting 0 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 1 monclient: found mon.noname-c 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:54.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.900+0000 7faa998fa640 1 -- 10.20.193.18:0/3696417538 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9406e8c0 con 0x7faa941527d0 2026-02-24T18:38:54.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.901+0000 7faa998fa640 1 --2- 10.20.193.18:0/3696417538 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa941527d0 0x7faa9406da60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa9406f370 tx=0x7faa8402ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85e5dedcd57f9620 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:54.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8403c070 con 0x7faa941527d0 2026-02-24T18:38:54.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:54.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:54.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: dump: 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: _finish_auth 0 2026-02-24T18:38:54.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: _check_auth_tickets 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8402fb70 con 0x7faa941527d0 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa9bb85640 5 monclient: authenticate success, global_id 4922 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa9bb85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 1 -- 10.20.193.18:0/3696417538 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84037440 con 0x7faa941527d0 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa9bb85640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa9bb85640 4 set_mon_vals no callback set 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.903+0000 7faa988f8640 10 monclient: dump: 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:54.915 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 10 monclient: shutdown 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 -- 10.20.193.18:0/3696417538 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa941527d0 msgr2=0x7faa9406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 --2- 10.20.193.18:0/3696417538 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa941527d0 0x7faa9406da60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa9406f370 tx=0x7faa8402ee60 comp rx=0 tx=0).stop 2026-02-24T18:38:54.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 -- 10.20.193.18:0/3696417538 shutdown_connections 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 --2- 10.20.193.18:0/3696417538 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa941527d0 0x7faa9406da60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 -- 10.20.193.18:0/3696417538 >> 10.20.193.18:0/3696417538 conn(0x7faa940cd780 msgr2=0x7faa940cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 -- 10.20.193.18:0/3696417538 shutdown_connections 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 -- 10.20.193.18:0/3696417538 wait complete. 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 10 monclient: build_initial_monmap 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 10 monclient: monmap: 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:54.917 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 librados: starting msgr at 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 librados: starting objecter 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.904+0000 7faa9bb85640 1 Processor -- start 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 -- start start 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 librados: setting wanted keys 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 librados: calling monclient init 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient: init 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:54.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 0x7faa94150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-a con 0x7faa940c6190 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa941527d0 0x7faa9414b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-b con 0x7faa941527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 0x7faa941f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): picked mon.noname-c con 0x7faa94152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7faa9406dfa0 con 0x7faa940c6190 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa9406ee80 con 0x7faa941527d0 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa940c9570 con 0x7faa94152bb0 2026-02-24T18:38:54.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9bb85640 10 monclient(hunting): authenticate will time out at 1678.586486s 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 0x7faa94150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9a0fb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 0x7faa941f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa990f9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa941527d0 0x7faa9414b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 0x7faa94150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:55064/0 (socket says 10.20.193.18:55064) 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 1 -- 10.20.193.18:0/4136515029 learned_addr learned my addr 10.20.193.18:0/4136515029 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 10 monclient(hunting): get_auth_request con 0x7faa940c6190 auth_method 0 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa998fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9a0fb640 10 monclient(hunting): get_auth_request con 0x7faa94152bb0 auth_method 0 2026-02-24T18:38:54.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9a0fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9a0fb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.905+0000 7faa9a0fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): get_auth_request con 0x7faa941527d0 auth_method 0 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa998fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa9a0fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa9a0fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa9a0fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:54.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.906+0000 7faa990f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 10 monclient(hunting): handle_auth_done global_id 4925 payload 824 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 -- 10.20.193.18:0/4136515029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa941527d0 msgr2=0x7faa9414b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa941527d0 0x7faa9414b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 -- 10.20.193.18:0/4136515029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 msgr2=0x7faa94150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 0x7faa94150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 10 monclient: _finish_hunting 0 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 monclient: found mon.noname-c 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.907+0000 7faa9a0fb640 1 -- 10.20.193.18:0/4136515029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9414a770 con 0x7faa94152bb0 2026-02-24T18:38:54.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.908+0000 7faa9a0fb640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 0x7faa941f6600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa9000cbc0 tx=0x7faa9000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.908+0000 7faa827fc640 10 client.?.objecter ms_handle_connect 0x7faa94152bb0 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.908+0000 7faa827fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9000acf0 con 0x7faa94152bb0 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 10 monclient: dump: 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:54.923 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 10 monclient: _finish_auth 0 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 10 monclient: _check_auth_tickets 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 5 monclient: authenticate success, global_id 4925 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 10 monclient: _renew_subs 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9414aa10 con 0x7faa94152bb0 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 10 client.4925.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:54.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 10 monclient: _renew_subs 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9414afd0 con 0x7faa94152bb0 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.909+0000 7faa9bb85640 1 librados: init done 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90003c80 con 0x7faa94152bb0 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900052d0 con 0x7faa94152bb0 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa83fff640 4 set_mon_vals no callback set 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa827fc640 10 monclient: dump: 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:54.925 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa9bb85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa9bb85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa9bb85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:54.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.910+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9406da60 con 0x7faa94152bb0 2026-02-24T18:38:54.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.921+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7faa90005470 con 0x7faa94152bb0 2026-02-24T18:38:54.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.922+0000 7faa827fc640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 0x7faa6407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:54.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.922+0000 7faa998fa640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 0x7faa6407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:54.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.922+0000 7faa998fa640 10 monclient: get_auth_request con 0x7faa64078e00 auth_method 0 2026-02-24T18:38:54.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.923+0000 7faa998fa640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 0x7faa6407b2c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa84016e30 tx=0x7faa84033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:54.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:54.923+0000 7faa827fc640 10 client.4925.objecter ms_handle_connect 0x7faa64078e00 2026-02-24T18:38:55.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.016+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7faa9005e900 con 0x7faa94152bb0 2026-02-24T18:38:55.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.016+0000 7faa827fc640 10 client.4925.objecter ms_dispatch2 0x7faa94000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:55.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.016+0000 7faa827fc640 3 client.4925.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:55.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.016+0000 7faa827fc640 3 client.4925.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:55.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.016+0000 7faa827fc640 20 client.4925.objecter dump_active .. 0 homeless 2026-02-24T18:38:55.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.022+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faa900628c0 con 0x7faa94152bb0 2026-02-24T18:38:55.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.022+0000 7faa827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:55.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.022+0000 7faa827fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:55.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.122+0000 7faa9bb85640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:55.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.122+0000 7faa9bb85640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:55.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.122+0000 7faa9bb85640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:38:55.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.122+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faa940c8150 con 0x7faa94152bb0 2026-02-24T18:38:55.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.125+0000 7faa827fc640 1 -- 10.20.193.18:0/4136515029 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7faa90066bc0 con 0x7faa94152bb0 2026-02-24T18:38:55.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.125+0000 7faa827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:55.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.125+0000 7faa827fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:55.127 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:55.127 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 10 librados: watch_flush enter 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 10 librados: watch_flush exit 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 20 client.4925.objecter shutdown clearing up homeless session... 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 10 client.4925.objecter successfully canceled tick 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 msgr2=0x7faa6407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 0x7faa6407b2c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa84016e30 tx=0x7faa84033000 comp rx=0 tx=0).stop 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 10 monclient: shutdown 2026-02-24T18:38:55.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 msgr2=0x7faa941f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 0x7faa941f6600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa9000cbc0 tx=0x7faa9000a430 comp rx=0 tx=0).stop 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 shutdown_connections 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7faa64078e00 0x7faa6407b2c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7faa941527d0 0x7faa9414b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7faa94152bb0 0x7faa941f6600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 --2- 10.20.193.18:0/4136515029 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7faa940c6190 0x7faa94150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 >> 10.20.193.18:0/4136515029 conn(0x7faa940cd780 msgr2=0x7faa940c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:55.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 shutdown_connections 2026-02-24T18:38:55.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 -- 10.20.193.18:0/4136515029 wait complete. 2026-02-24T18:38:55.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:55.126+0000 7faa9bb85640 1 librados: shutdown 2026-02-24T18:38:58.140 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:38:58.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 10 monclient: get_monmap_and_config 2026-02-24T18:38:58.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 10 monclient: build_initial_monmap 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 10 monclient: monmap: 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:38:58.209233+0000 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:38:58.209233+0000 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:38:58.211 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:38:58.212 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 1 Processor -- start 2026-02-24T18:38:58.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 1 -- start start 2026-02-24T18:38:58.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.208+0000 7f50582f3640 10 monclient: init 2026-02-24T18:38:58.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:58.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient: _add_conns ranks=[1,2,4,0,5,3] 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f505006ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f50500c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f505015a5e0 0x7f5050070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-b con 0x7f505015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50500c7e90 con 0x7f505006ff20 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50500c8170 con 0x7f50500c99a0 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5050159a40 con 0x7f505015a5e0 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f50582f3640 10 monclient(hunting): authenticate will time out at 1681.890371s 2026-02-24T18:38:58.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f5055867640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f50500c99a0 0x7f505014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:48688/0 (socket says 10.20.193.18:48688) 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.209+0000 7f5055867640 1 -- 10.20.193.18:0/966232310 learned_addr learned my addr 10.20.193.18:0/966232310 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 1 --2- 10.20.193.18:0/966232310 >> v2:10.20.193.32:3300/0 conn(0x7f505015a5e0 0x7f5050070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): get_auth_request con 0x7f505015a5e0 auth_method 0 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.210+0000 7f5055066640 1 -- 10.20.193.18:0/966232310 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325074621 0 0) 0x7f50500c8170 con 0x7f50500c99a0 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5055066640 10 monclient(hunting): _negotiate 2026-02-24T18:38:58.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5055066640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5055066640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5055066640 10 monclient(hunting): my global_id is 4928 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5055066640 1 -- 10.20.193.18:0/966232310 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5038003620 con 0x7f50500c99a0 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 10 monclient(hunting): handle_auth_done global_id 4966 payload 274 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 1 -- 10.20.193.18:0/966232310 >> v1:10.20.193.110:6789/0 conn(0x7f50500c99a0 legacy=0x7f505014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 1 -- 10.20.193.18:0/966232310 >> v1:10.20.193.32:6789/0 conn(0x7f505006ff20 legacy=0x7f5050159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 10 monclient: _finish_hunting 0 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 1 monclient: found mon.noname-b 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.211+0000 7f5056869640 1 -- 10.20.193.18:0/966232310 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50500c8450 con 0x7f505015a5e0 2026-02-24T18:38:58.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.213+0000 7f5056869640 1 --2- 10.20.193.18:0/966232310 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505015a5e0 0x7f5050070300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f504c009dc0 tx=0x7f504c02f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1921cadd0e941650 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 1 -- 10.20.193.18:0/966232310 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c03c070 con 0x7f505015a5e0 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: dump: 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.225 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: _finish_auth 0 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: _check_auth_tickets 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 1 -- 10.20.193.18:0/966232310 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504c037440 con 0x7f505015a5e0 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 1 -- 10.20.193.18:0/966232310 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c037810 con 0x7f505015a5e0 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:58.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f5055066640 10 monclient: dump: 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:58.227 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 5 monclient: authenticate success, global_id 4966 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 10 monclient: get_monmap_and_config success 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 4 set_mon_vals no callback set 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 10 monclient: shutdown 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 -- 10.20.193.18:0/966232310 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505015a5e0 msgr2=0x7f5050070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 --2- 10.20.193.18:0/966232310 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505015a5e0 0x7f5050070300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f504c009dc0 tx=0x7f504c02f230 comp rx=0 tx=0).stop 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 -- 10.20.193.18:0/966232310 shutdown_connections 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 --2- 10.20.193.18:0/966232310 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505015a5e0 0x7f5050070300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 -- 10.20.193.18:0/966232310 >> 10.20.193.18:0/966232310 conn(0x7f50500cd780 msgr2=0x7f50500cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:58.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 -- 10.20.193.18:0/966232310 shutdown_connections 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.215+0000 7f50582f3640 1 -- 10.20.193.18:0/966232310 wait complete. 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 10 monclient: build_initial_monmap 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 10 monclient: monmap: 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:38:58.229 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 librados: starting msgr at 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 librados: starting objecter 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 Processor -- start 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 -- start start 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 librados: setting wanted keys 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 librados: calling monclient init 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 10 monclient: init 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.216+0000 7f50582f3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:58.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-b con 0x7f505006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50500c99a0 0x7f50501f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-a con 0x7f50500c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f505015a5e0 0x7f50501f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): picked mon.noname-c con 0x7f505015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5050150520 con 0x7f50500c99a0 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50500c8a10 con 0x7f505006ff20 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50501506a0 con 0x7f505015a5e0 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f50582f3640 10 monclient(hunting): authenticate will time out at 1681.898047s 2026-02-24T18:38:58.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50500c99a0 0x7f50501f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056869640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f505015a5e0 0x7f50501f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:33974/0 (socket says 10.20.193.18:33974) 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 1 -- 10.20.193.18:0/2865444803 learned_addr learned my addr 10.20.193.18:0/2865444803 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): get_auth_request con 0x7f505006ff20 auth_method 0 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): get_auth_request con 0x7f50500c99a0 auth_method 0 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:58.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056869640 10 monclient(hunting): get_auth_request con 0x7f505015a5e0 auth_method 0 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056869640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056869640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5055867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.217+0000 7f5056068640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.218+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.218+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:38:58.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.218+0000 7f5056869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 10 monclient(hunting): handle_auth_done global_id 4969 payload 824 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 -- 10.20.193.18:0/2865444803 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f505015a5e0 msgr2=0x7f50501f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f505015a5e0 0x7f50501f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 -- 10.20.193.18:0/2865444803 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50500c99a0 msgr2=0x7f50501f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50500c99a0 0x7f50501f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 10 monclient: _finish_hunting 0 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 monclient: found mon.noname-b 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.219+0000 7f5056068640 1 -- 10.20.193.18:0/2865444803 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50501f3e60 con 0x7f505006ff20 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.220+0000 7f5056068640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f504000c5d0 tx=0x7f504000caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:58.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.220+0000 7f5036ffd640 10 client.?.objecter ms_handle_connect 0x7f505006ff20 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.220+0000 7f5036ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.221+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040016020 con 0x7f505006ff20 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.221+0000 7f5036ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.221+0000 7f5036ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.221+0000 7f5036ffd640 10 monclient: dump: 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:58.235 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: _finish_auth 0 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 5 monclient: authenticate success, global_id 4969 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040012850 con 0x7f505006ff20 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040012c40 con 0x7f505006ff20 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _renew_subs 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50501f4140 con 0x7f505006ff20 2026-02-24T18:38:58.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5054865640 4 set_mon_vals no callback set 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f5036ffd640 10 monclient: dump: 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:38:58.237 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 client.4969.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _renew_subs 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50500c1570 con 0x7f505006ff20 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 1 librados: init done 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.222+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5050070300 con 0x7f505006ff20 2026-02-24T18:38:58.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.233+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f50400068d0 con 0x7f505006ff20 2026-02-24T18:38:58.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.234+0000 7f5036ffd640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 0x7f502007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:38:58.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.234+0000 7f5055867640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 0x7f502007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:38:58.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.234+0000 7f5055867640 10 monclient: get_auth_request con 0x7f5020078e80 auth_method 0 2026-02-24T18:38:58.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.235+0000 7f5036ffd640 10 client.4969.objecter ms_handle_connect 0x7f5020078e80 2026-02-24T18:38:58.240 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.235+0000 7f5055867640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 0x7f502007b340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5044007b60 tx=0x7f5044007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:38:58.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.327+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f5040012de0 con 0x7f505006ff20 2026-02-24T18:38:58.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.327+0000 7f5036ffd640 10 client.4969.objecter ms_dispatch2 0x7f5050000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:38:58.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.327+0000 7f5036ffd640 3 client.4969.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:38:58.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.327+0000 7f5036ffd640 3 client.4969.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:38:58.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.327+0000 7f5036ffd640 20 client.4969.objecter dump_active .. 0 homeless 2026-02-24T18:38:58.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.334+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5040020030 con 0x7f505006ff20 2026-02-24T18:38:58.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.334+0000 7f5036ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:38:58.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.334+0000 7f5036ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:38:58.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.434+0000 7f50582f3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:38:58.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.434+0000 7f50582f3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:58.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.434+0000 7f50582f3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:38:58.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.434+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f50500c4c50 con 0x7f505006ff20 2026-02-24T18:38:58.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.437+0000 7f5036ffd640 1 -- 10.20.193.18:0/2865444803 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5040061de0 con 0x7f505006ff20 2026-02-24T18:38:58.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.437+0000 7f5036ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:38:58.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.437+0000 7f5036ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:38:58.439 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:38:58.439 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:38:58.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 10 librados: watch_flush enter 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 10 librados: watch_flush exit 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 20 client.4969.objecter shutdown clearing up homeless session... 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 10 client.4969.objecter successfully canceled tick 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 msgr2=0x7f502007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 0x7f502007b340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5044007b60 tx=0x7f5044007ab0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 10 monclient: shutdown 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:38:58.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 msgr2=0x7f50501f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f504000c5d0 tx=0x7f504000caa0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 shutdown_connections 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5020078e80 0x7f502007b340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f505015a5e0 0x7f50501f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50500c99a0 0x7f50501f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 --2- 10.20.193.18:0/2865444803 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f505006ff20 0x7f50501f7510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 >> 10.20.193.18:0/2865444803 conn(0x7f50500cd780 msgr2=0x7f5050073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 shutdown_connections 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 -- 10.20.193.18:0/2865444803 wait complete. 2026-02-24T18:38:58.443 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:38:58.438+0000 7f50582f3640 1 librados: shutdown 2026-02-24T18:38:59.594 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:37.540 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.596 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:37.542 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.605 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:37.551 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.625 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:37.533 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.625 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:37.571 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.626 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:37.572 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.627 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:37.535 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.635 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:37.543 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:38:59.637 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:37.582 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:01.450 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 10 monclient: get_monmap_and_config 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 10 monclient: build_initial_monmap 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 10 monclient: monmap: 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:01.517938+0000 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:01.517938+0000 2026-02-24T18:39:01.519 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 1 Processor -- start 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 1 -- start start 2026-02-24T18:39:01.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.517+0000 7f556ed3e640 10 monclient: init 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient: _add_conns ranks=[3,5,4,0,1,2] 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f556806ff20 0x7f5568070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-a con 0x7f556806ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f55680c99a0 0x7f5568159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-c con 0x7f55680c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f556815a410 0x7f556814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-b con 0x7f556815a410 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f5568159a40 con 0x7f556806ff20 2026-02-24T18:39:01.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f556815a860 con 0x7f556815a410 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f55680c7e90 con 0x7f55680c99a0 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556ed3e640 10 monclient(hunting): authenticate will time out at 1685.199085s 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f556806ff20 0x7f5568070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556d2b4640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f556815a410 0x7f556814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f556806ff20 0x7f5568070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50522/0 (socket says 10.20.193.18:50522) 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 1 -- 10.20.193.18:0/1594286994 learned_addr learned my addr 10.20.193.18:0/1594286994 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): get_auth_request con 0x7f556806ff20 auth_method 0 2026-02-24T18:39:01.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f555ffff640 1 --2- 10.20.193.18:0/1594286994 >> v2:10.20.193.110:3300/0 conn(0x7f55680c99a0 0x7f5568159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f555ffff640 10 monclient(hunting): get_auth_request con 0x7f55680c99a0 auth_method 0 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f555ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f555ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f555ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556d2b4640 10 monclient(hunting): get_auth_request con 0x7f556815a410 auth_method 0 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556d2b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556d2b4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.518+0000 7f556d2b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 10 monclient(hunting): handle_auth_done global_id 4992 payload 274 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 -- 10.20.193.18:0/1594286994 >> v2:10.20.193.110:3300/0 conn(0x7f55680c99a0 msgr2=0x7f5568159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 --2- 10.20.193.18:0/1594286994 >> v2:10.20.193.110:3300/0 conn(0x7f55680c99a0 0x7f5568159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 -- 10.20.193.18:0/1594286994 >> v2:10.20.193.32:3300/0 conn(0x7f556815a410 msgr2=0x7f556814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 --2- 10.20.193.18:0/1594286994 >> v2:10.20.193.32:3300/0 conn(0x7f556815a410 0x7f556814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.527 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 10 monclient: _finish_hunting 0 2026-02-24T18:39:01.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 monclient: found mon.noname-a 2026-02-24T18:39:01.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:01.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.519+0000 7f556cab3640 1 -- 10.20.193.18:0/1594286994 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55680c8170 con 0x7f556806ff20 2026-02-24T18:39:01.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.520+0000 7f556cab3640 1 --2- 10.20.193.18:0/1594286994 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556806ff20 0x7f5568070300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55680c5380 tx=0x7f555802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2ce6ccd3175784 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:01.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 1 -- 10.20.193.18:0/1594286994 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555803c070 con 0x7f556806ff20 2026-02-24T18:39:01.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: dump: 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:01.535 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f556ed3e640 5 monclient: authenticate success, global_id 4992 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f556ed3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 1 -- 10.20.193.18:0/1594286994 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555802fb50 con 0x7f556806ff20 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 1 -- 10.20.193.18:0/1594286994 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558037440 con 0x7f556806ff20 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f555f7fe640 10 monclient: dump: 2026-02-24T18:39:01.536 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f556ed3e640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.522+0000 7f556ed3e640 4 set_mon_vals no callback set 2026-02-24T18:39:01.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 10 monclient: shutdown 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 -- 10.20.193.18:0/1594286994 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556806ff20 msgr2=0x7f5568070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 --2- 10.20.193.18:0/1594286994 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556806ff20 0x7f5568070300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55680c5380 tx=0x7f555802eed0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 -- 10.20.193.18:0/1594286994 shutdown_connections 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 --2- 10.20.193.18:0/1594286994 >> v2:10.20.193.32:3300/0 conn(0x7f556815a410 0x7f556814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 --2- 10.20.193.18:0/1594286994 >> v2:10.20.193.110:3300/0 conn(0x7f55680c99a0 0x7f5568159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 --2- 10.20.193.18:0/1594286994 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556806ff20 0x7f5568070300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 -- 10.20.193.18:0/1594286994 >> 10.20.193.18:0/1594286994 conn(0x7f55680cd780 msgr2=0x7f55680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:01.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 -- 10.20.193.18:0/1594286994 shutdown_connections 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 -- 10.20.193.18:0/1594286994 wait complete. 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 10 monclient: build_initial_monmap 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 10 monclient: monmap: 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:01.539 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 librados: starting msgr at 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.523+0000 7f556ed3e640 1 librados: starting objecter 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 Processor -- start 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 -- start start 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 librados: setting wanted keys 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 librados: calling monclient init 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient: init 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f556806ff20 0x7f5568152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-c con 0x7f556806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:01.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f55680c99a0 0x7f55681f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-b con 0x7f55680c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): picked mon.noname-a con 0x7f556815a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55680c8a10 con 0x7f556815a410 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55680c8730 con 0x7f55680c99a0 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5568150350 con 0x7f556806ff20 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.524+0000 7f556ed3e640 10 monclient(hunting): authenticate will time out at 1685.205837s 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f556806ff20 0x7f5568152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f55680c99a0 0x7f55681f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50524/0 (socket says 10.20.193.18:50524) 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 1 -- 10.20.193.18:0/561589226 learned_addr learned my addr 10.20.193.18:0/561589226 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): get_auth_request con 0x7f556815a410 auth_method 0 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): get_auth_request con 0x7f55680c99a0 auth_method 0 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556d2b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): get_auth_request con 0x7f556806ff20 auth_method 0 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f555ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.525+0000 7f556cab3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:01.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 10 monclient(hunting): handle_auth_done global_id 4995 payload 824 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 -- 10.20.193.18:0/561589226 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f556806ff20 msgr2=0x7f5568152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f556806ff20 0x7f5568152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 -- 10.20.193.18:0/561589226 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f55680c99a0 msgr2=0x7f55681f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f55680c99a0 0x7f55681f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 10 monclient: _finish_hunting 0 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 monclient: found mon.noname-a 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.526+0000 7f556d2b4640 1 -- 10.20.193.18:0/561589226 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681532a0 con 0x7f556815a410 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.528+0000 7f556d2b4640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f556400efc0 tx=0x7f556400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.528+0000 7f555d7fa640 10 client.?.objecter ms_handle_connect 0x7f556815a410 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.528+0000 7f555d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:01.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564018020 con 0x7f556815a410 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: dump: 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:01.545 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5564014480 con 0x7f556815a410 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564014870 con 0x7f556815a410 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555d7fa640 10 monclient: dump: 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.546 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f555effd640 4 set_mon_vals no callback set 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 5 monclient: authenticate success, global_id 4995 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _renew_subs 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681519e0 con 0x7f556815a410 2026-02-24T18:39:01.547 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 client.4995.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _renew_subs 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5568151fa0 con 0x7f556815a410 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 1 librados: init done 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.548 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.530+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55681506e0 con 0x7f556815a410 2026-02-24T18:39:01.549 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.542+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5564014a10 con 0x7f556815a410 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.542+0000 7f555d7fa640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 0x7f553807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.543+0000 7f556cab3640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 0x7f553807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.543+0000 7f556cab3640 10 monclient: get_auth_request con 0x7f5538078e30 auth_method 0 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.543+0000 7f555d7fa640 10 client.4995.objecter ms_handle_connect 0x7f5538078e30 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.543+0000 7f556cab3640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 0x7f553807b2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55680709c0 tx=0x7f5558039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.545+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f556405ede0 con 0x7f556815a410 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.545+0000 7f555d7fa640 10 client.4995.objecter ms_dispatch2 0x7f5568000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.545+0000 7f555d7fa640 3 client.4995.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.545+0000 7f555d7fa640 3 client.4995.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:01.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.545+0000 7f555d7fa640 20 client.4995.objecter dump_active .. 0 homeless 2026-02-24T18:39:01.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.647+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5564022030 con 0x7f556815a410 2026-02-24T18:39:01.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.647+0000 7f555d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:01.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.647+0000 7f555d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:01.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.748+0000 7f556ed3e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:01.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.748+0000 7f556ed3e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:01.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.748+0000 7f556ed3e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:01.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.748+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5568073e20 con 0x7f556815a410 2026-02-24T18:39:01.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.751+0000 7f555d7fa640 1 -- 10.20.193.18:0/561589226 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5564010040 con 0x7f556815a410 2026-02-24T18:39:01.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.751+0000 7f555d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:01.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.751+0000 7f555d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:01.753 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:01.754 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 10 librados: watch_flush enter 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 10 librados: watch_flush exit 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 20 client.4995.objecter shutdown clearing up homeless session... 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 10 client.4995.objecter successfully canceled tick 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 msgr2=0x7f553807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:01.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 0x7f553807b2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55680709c0 tx=0x7f5558039040 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 10 monclient: shutdown 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 msgr2=0x7f55681f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f556400efc0 tx=0x7f556400c490 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 shutdown_connections 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5538078e30 0x7f553807b2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f55680c99a0 0x7f55681f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f556815a410 0x7f55681f6540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 --2- 10.20.193.18:0/561589226 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f556806ff20 0x7f5568152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 >> 10.20.193.18:0/561589226 conn(0x7f55680cd780 msgr2=0x7f55680cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 shutdown_connections 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 -- 10.20.193.18:0/561589226 wait complete. 2026-02-24T18:39:01.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:01.752+0000 7f556ed3e640 1 librados: shutdown 2026-02-24T18:39:04.764 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:04.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 10 monclient: get_monmap_and_config 2026-02-24T18:39:04.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 10 monclient: build_initial_monmap 2026-02-24T18:39:04.834 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 10 monclient: monmap: 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:04.833316+0000 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:04.833316+0000 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:04.835 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:04.836 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 1 Processor -- start 2026-02-24T18:39:04.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 1 -- start start 2026-02-24T18:39:04.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.832+0000 7f1c15c88640 10 monclient: init 2026-02-24T18:39:04.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient: _add_conns ranks=[0,5,2,1,4,3] 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1c1006ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1c10155eb0 0x7f1c10070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-c con 0x7f1c10155eb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c1015c970 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c100725b0 con 0x7f1c1006ff20 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c10072890 con 0x7f1c1015c970 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1c10072b70 con 0x7f1c10155eb0 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:04.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c15c88640 10 monclient(hunting): authenticate will time out at 1688.514463s 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c0f7fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1c1006ff20 0x7f1c101557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37552/0 (socket says 10.20.193.18:37552) 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c0f7fe640 1 -- 10.20.193.18:0/125140031 learned_addr learned my addr 10.20.193.18:0/125140031 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.833+0000 7f1c0effd640 1 --2- 10.20.193.18:0/125140031 >> v2:10.20.193.110:3300/0 conn(0x7f1c10155eb0 0x7f1c10070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): get_auth_request con 0x7f1c10155eb0 auth_method 0 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0e7fc640 1 -- 10.20.193.18:0/125140031 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365755580 0 0) 0x7f1c100725b0 con 0x7f1c1006ff20 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0e7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:39:04.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.834+0000 7f1c0e7fc640 10 monclient(hunting): my global_id is 4998 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0e7fc640 1 -- 10.20.193.18:0/125140031 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bf4003620 con 0x7f1c1006ff20 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 10 monclient(hunting): handle_auth_done global_id 4940 payload 274 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 1 -- 10.20.193.18:0/125140031 >> v1:10.20.193.110:6789/0 conn(0x7f1c1015c970 legacy=0x7f1c100c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 1 -- 10.20.193.18:0/125140031 >> v1:10.20.193.18:6789/0 conn(0x7f1c1006ff20 legacy=0x7f1c101557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 10 monclient: _finish_hunting 0 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 1 monclient: found mon.noname-c 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.835+0000 7f1c0effd640 1 -- 10.20.193.18:0/125140031 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c10072e50 con 0x7f1c10155eb0 2026-02-24T18:39:04.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.836+0000 7f1c0effd640 1 --2- 10.20.193.18:0/125140031 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c10070300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098a0 tx=0x7f1c0002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d4188f6fe0a693f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 1 -- 10.20.193.18:0/125140031 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0003c070 con 0x7f1c10155eb0 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: dump: 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.846 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 1 -- 10.20.193.18:0/125140031 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0002fad0 con 0x7f1c10155eb0 2026-02-24T18:39:04.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 5 monclient: authenticate success, global_id 4940 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 1 -- 10.20.193.18:0/125140031 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00037440 con 0x7f1c10155eb0 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 4 set_mon_vals no callback set 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c0e7fc640 10 monclient: dump: 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:04.848 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 10 monclient: shutdown 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:04.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 1 -- 10.20.193.18:0/125140031 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 msgr2=0x7f1c10070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.838+0000 7f1c15c88640 1 --2- 10.20.193.18:0/125140031 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c10070300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098a0 tx=0x7f1c0002ee10 comp rx=0 tx=0).stop 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 -- 10.20.193.18:0/125140031 shutdown_connections 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 --2- 10.20.193.18:0/125140031 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c10070300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 -- 10.20.193.18:0/125140031 >> 10.20.193.18:0/125140031 conn(0x7f1c100cd780 msgr2=0x7f1c100cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 -- 10.20.193.18:0/125140031 shutdown_connections 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 -- 10.20.193.18:0/125140031 wait complete. 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 10 monclient: build_initial_monmap 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 10 monclient: monmap: 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:04.850 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 librados: starting msgr at 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 librados: starting objecter 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 Processor -- start 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 -- start start 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 librados: setting wanted keys 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 1 librados: calling monclient init 2026-02-24T18:39:04.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.839+0000 7f1c15c88640 10 monclient: init 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c1006ff20 0x7f1c100c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-b con 0x7f1c1006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c101f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-c con 0x7f1c10155eb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): picked mon.noname-a con 0x7f1c1015c970 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1015d340 con 0x7f1c1015c970 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:04.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1015d1c0 con 0x7f1c1006ff20 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1015d4c0 con 0x7f1c10155eb0 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c15c88640 10 monclient(hunting): authenticate will time out at 1688.521150s 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c101f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50530/0 (socket says 10.20.193.18:50530) 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 1 -- 10.20.193.18:0/1380410107 learned_addr learned my addr 10.20.193.18:0/1380410107 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): get_auth_request con 0x7f1c1015c970 auth_method 0 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:04.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0f7fe640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c1006ff20 0x7f1c100c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): get_auth_request con 0x7f1c10155eb0 auth_method 0 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0f7fe640 10 monclient(hunting): get_auth_request con 0x7f1c1006ff20 auth_method 0 2026-02-24T18:39:04.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.840+0000 7f1c0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.841+0000 7f1c0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.841+0000 7f1c0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.841+0000 7f1c0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 10 monclient(hunting): handle_auth_done global_id 5001 payload 824 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 -- 10.20.193.18:0/1380410107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 msgr2=0x7f1c101f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c101f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 -- 10.20.193.18:0/1380410107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c1006ff20 msgr2=0x7f1c100c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c1006ff20 0x7f1c100c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 monclient: found mon.noname-a 2026-02-24T18:39:04.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:04.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:04.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.842+0000 7f1c0ffff640 1 -- 10.20.193.18:0/1380410107 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c100c5260 con 0x7f1c1015c970 2026-02-24T18:39:04.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.843+0000 7f1c0ffff640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400b9c0 tx=0x7f1c0400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:04.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.843+0000 7f1bebfff640 10 client.?.objecter ms_handle_connect 0x7f1c1015c970 2026-02-24T18:39:04.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.843+0000 7f1bebfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:04.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400c790 con 0x7f1c1015c970 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: dump: 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:04.859 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: _finish_auth 0 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: _check_auth_tickets 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 5 monclient: authenticate success, global_id 5001 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c04010070 con 0x7f1c1015c970 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400cb70 con 0x7f1c1015c970 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 10 monclient: _renew_subs 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c100c5540 con 0x7f1c1015c970 2026-02-24T18:39:04.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c0dffb640 4 set_mon_vals no callback set 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1bebfff640 10 monclient: dump: 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:04.861 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 10 client.5001.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 10 monclient: _renew_subs 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c100c5b00 con 0x7f1c1015c970 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.844+0000 7f1c15c88640 1 librados: init done 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.845+0000 7f1c15c88640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.845+0000 7f1c15c88640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.845+0000 7f1c15c88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.845+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c10070300 con 0x7f1c1015c970 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.856+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1c04004070 con 0x7f1c1015c970 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 0x7f1be007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:04.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1c0f7fe640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 0x7f1be007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:04.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1c0f7fe640 10 monclient: get_auth_request con 0x7f1be0078e80 auth_method 0 2026-02-24T18:39:04.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f1c0400cd10 con 0x7f1c1015c970 2026-02-24T18:39:04.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 10 client.5001.objecter ms_dispatch2 0x7f1c10000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:04.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 3 client.5001.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:04.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 3 client.5001.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:04.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 20 client.5001.objecter dump_active .. 0 homeless 2026-02-24T18:39:04.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1bebfff640 10 client.5001.objecter ms_handle_connect 0x7f1be0078e80 2026-02-24T18:39:04.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.857+0000 7f1c0f7fe640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 0x7f1be007b340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007b60 tx=0x7f1bfc007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:04.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.956+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c04061b70 con 0x7f1c1015c970 2026-02-24T18:39:04.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.956+0000 7f1bebfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:04.958 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:04.956+0000 7f1bebfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:05.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.057+0000 7f1c15c88640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:05.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.057+0000 7f1c15c88640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:05.059 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.057+0000 7f1c15c88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:05.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.057+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1c1015c750 con 0x7f1c1015c970 2026-02-24T18:39:05.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.059+0000 7f1bebfff640 1 -- 10.20.193.18:0/1380410107 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1c04065e70 con 0x7f1c1015c970 2026-02-24T18:39:05.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.059+0000 7f1bebfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:05.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.059+0000 7f1bebfff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:05.062 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:05.062 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:05.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 10 librados: watch_flush enter 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 10 librados: watch_flush exit 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 20 client.5001.objecter shutdown clearing up homeless session... 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 10 client.5001.objecter successfully canceled tick 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 msgr2=0x7f1be007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 0x7f1be007b340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007b60 tx=0x7f1bfc007ab0 comp rx=0 tx=0).stop 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 10 monclient: shutdown 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 msgr2=0x7f1c101f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400b9c0 tx=0x7f1c0400be90 comp rx=0 tx=0).stop 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 shutdown_connections 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1be0078e80 0x7f1be007b340 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:05.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1c1015c970 0x7f1c101f5160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1c10155eb0 0x7f1c101f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 --2- 10.20.193.18:0/1380410107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1c1006ff20 0x7f1c100c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 >> 10.20.193.18:0/1380410107 conn(0x7f1c100cd780 msgr2=0x7f1c1015e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 shutdown_connections 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 -- 10.20.193.18:0/1380410107 wait complete. 2026-02-24T18:39:05.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:05.061+0000 7f1c15c88640 1 librados: shutdown 2026-02-24T18:39:08.073 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:08.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 10 monclient: get_monmap_and_config 2026-02-24T18:39:08.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 10 monclient: build_initial_monmap 2026-02-24T18:39:08.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 10 monclient: monmap: 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:08.142342+0000 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:08.142342+0000 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:08.144 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:08.145 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 1 Processor -- start 2026-02-24T18:39:08.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 1 -- start start 2026-02-24T18:39:08.145 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.141+0000 7fa943ded640 10 monclient: init 2026-02-24T18:39:08.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient: _add_conns ranks=[4,0,1,2,5,3] 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa93c06ff20 0x7fa93c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-b con 0x7fa93c06ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa93c0c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa93c15a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa93c0c7e90 con 0x7fa93c0c99a0 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa93c0c8170 con 0x7fa93c15a5e0 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:08.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa93c159a40 con 0x7fa93c06ff20 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa943ded640 10 monclient(hunting): authenticate will time out at 1691.823485s 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa941361640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa93c0c99a0 0x7fa93c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37568/0 (socket says 10.20.193.18:37568) 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa941361640 1 -- 10.20.193.18:0/3396161870 learned_addr learned my addr 10.20.193.18:0/3396161870 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.142+0000 7fa941b62640 1 --2- 10.20.193.18:0/3396161870 >> v2:10.20.193.32:3300/0 conn(0x7fa93c06ff20 0x7fa93c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): get_auth_request con 0x7fa93c06ff20 auth_method 0 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:08.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.143+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 1 -- 10.20.193.18:0/3396161870 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128035143 0 0) 0x7fa93c0c7e90 con 0x7fa93c0c99a0 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 10 monclient(hunting): _negotiate 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 10 monclient(hunting): my global_id is 5004 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa940b60640 1 -- 10.20.193.18:0/3396161870 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa924003620 con 0x7fa93c0c99a0 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 10 monclient(hunting): handle_auth_done global_id 4981 payload 274 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 1 -- 10.20.193.18:0/3396161870 >> v1:10.20.193.32:6789/0 conn(0x7fa93c15a5e0 legacy=0x7fa93c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 1 -- 10.20.193.18:0/3396161870 >> v1:10.20.193.18:6789/0 conn(0x7fa93c0c99a0 legacy=0x7fa93c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 10 monclient: _finish_hunting 0 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 1 monclient: found mon.noname-b 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:08.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.144+0000 7fa941b62640 1 -- 10.20.193.18:0/3396161870 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c0c8450 con 0x7fa93c06ff20 2026-02-24T18:39:08.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.145+0000 7fa941b62640 1 --2- 10.20.193.18:0/3396161870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c06ff20 0x7fa93c070300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0098d0 tx=0x7fa92c02ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e67e3c193948d55 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:08.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 1 -- 10.20.193.18:0/3396161870 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c03c070 con 0x7fa93c06ff20 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: dump: 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:08.158 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: _finish_auth 0 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: _check_auth_tickets 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 1 -- 10.20.193.18:0/3396161870 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c02fb00 con 0x7fa93c06ff20 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 5 monclient: authenticate success, global_id 4981 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 1 -- 10.20.193.18:0/3396161870 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c037440 con 0x7fa93c06ff20 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 4 set_mon_vals no callback set 2026-02-24T18:39:08.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa940b60640 10 monclient: dump: 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:08.160 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 10 monclient: shutdown 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 -- 10.20.193.18:0/3396161870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c06ff20 msgr2=0x7fa93c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 --2- 10.20.193.18:0/3396161870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c06ff20 0x7fa93c070300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0098d0 tx=0x7fa92c02ee80 comp rx=0 tx=0).stop 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 -- 10.20.193.18:0/3396161870 shutdown_connections 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 --2- 10.20.193.18:0/3396161870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c06ff20 0x7fa93c070300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 -- 10.20.193.18:0/3396161870 >> 10.20.193.18:0/3396161870 conn(0x7fa93c0cd780 msgr2=0x7fa93c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 -- 10.20.193.18:0/3396161870 shutdown_connections 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 1 -- 10.20.193.18:0/3396161870 wait complete. 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 10 monclient: build_initial_monmap 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.148+0000 7fa943ded640 10 monclient: monmap: 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:08.161 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 librados: starting msgr at 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 librados: starting objecter 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 Processor -- start 2026-02-24T18:39:08.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 -- start start 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 librados: setting wanted keys 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 librados: calling monclient init 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient: init 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa93c06ff20 0x7fa93c1f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-c con 0x7fa93c06ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 0x7fa93c1f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-a con 0x7fa93c0c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 0x7fa93c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): picked mon.noname-b con 0x7fa93c15a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:08.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c150520 con 0x7fa93c0c99a0 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c0c8a10 con 0x7fa93c15a5e0 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c1506a0 con 0x7fa93c06ff20 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa943ded640 10 monclient(hunting): authenticate will time out at 1691.830699s 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.149+0000 7fa941361640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 0x7fa93c1f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 0x7fa93c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 0x7fa93c1f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:50540/0 (socket says 10.20.193.18:50540) 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 1 -- 10.20.193.18:0/4263646820 learned_addr learned my addr 10.20.193.18:0/4263646820 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): get_auth_request con 0x7fa93c0c99a0 auth_method 0 2026-02-24T18:39:08.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): get_auth_request con 0x7fa93c15a5e0 auth_method 0 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa93c06ff20 0x7fa93c1f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941361640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:08.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa942363640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): get_auth_request con 0x7fa93c06ff20 auth_method 0 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.150+0000 7fa941b62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 10 monclient(hunting): handle_auth_done global_id 4984 payload 824 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 -- 10.20.193.18:0/4263646820 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa93c06ff20 msgr2=0x7fa93c1f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa93c06ff20 0x7fa93c1f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 -- 10.20.193.18:0/4263646820 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 msgr2=0x7fa93c1f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 0x7fa93c1f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 10 monclient: _finish_hunting 0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 monclient: found mon.noname-b 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.151+0000 7fa942363640 1 -- 10.20.193.18:0/4263646820 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c06d700 con 0x7fa93c15a5e0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.152+0000 7fa942363640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 0x7fa93c154d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa93800be20 tx=0x7fa93800bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.152+0000 7fa92a7fc640 10 client.?.objecter ms_handle_connect 0x7fa93c15a5e0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.152+0000 7fa92a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938002a60 con 0x7fa93c15a5e0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: dump: 2026-02-24T18:39:08.167 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:39:08.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938010070 con 0x7fa93c15a5e0 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938002e40 con 0x7fa93c15a5e0 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92a7fc640 10 monclient: dump: 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:08.169 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 5 monclient: authenticate success, global_id 4984 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa92bfff640 4 set_mon_vals no callback set 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _renew_subs 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c06d9e0 con 0x7fa93c15a5e0 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 client.4984.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _renew_subs 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c06dfa0 con 0x7fa93c15a5e0 2026-02-24T18:39:08.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 1 librados: init done 2026-02-24T18:39:08.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:08.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:08.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.154+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c070300 con 0x7fa93c15a5e0 2026-02-24T18:39:08.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.165+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa938004070 con 0x7fa93c15a5e0 2026-02-24T18:39:08.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.165+0000 7fa92a7fc640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 0x7fa90c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:08.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.165+0000 7fa941b62640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 0x7fa90c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:08.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.165+0000 7fa941b62640 10 monclient: get_auth_request con 0x7fa90c078e00 auth_method 0 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 10 client.4984.objecter ms_handle_connect 0x7fa90c078e00 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa941b62640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 0x7fa90c07b2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa92c039000 tx=0x7fa92c033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa93805dc50 con 0x7fa93c15a5e0 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 10 client.4984.objecter ms_dispatch2 0x7fa93c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 3 client.4984.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 3 client.4984.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:08.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.166+0000 7fa92a7fc640 20 client.4984.objecter dump_active .. 0 homeless 2026-02-24T18:39:08.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.264+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa938061c10 con 0x7fa93c15a5e0 2026-02-24T18:39:08.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.264+0000 7fa92a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:08.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.264+0000 7fa92a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:08.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.363+0000 7fa943ded640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:08.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.363+0000 7fa943ded640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:08.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.363+0000 7fa943ded640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:08.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.363+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa93c14f440 con 0x7fa93c15a5e0 2026-02-24T18:39:08.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa92a7fc640 1 -- 10.20.193.18:0/4263646820 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa938065f10 con 0x7fa93c15a5e0 2026-02-24T18:39:08.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa92a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:08.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa92a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:08.369 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:08.369 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:08.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 10 librados: watch_flush enter 2026-02-24T18:39:08.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 10 librados: watch_flush exit 2026-02-24T18:39:08.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 20 client.4984.objecter shutdown clearing up homeless session... 2026-02-24T18:39:08.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 10 client.4984.objecter successfully canceled tick 2026-02-24T18:39:08.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 msgr2=0x7fa90c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 0x7fa90c07b2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa92c039000 tx=0x7fa92c033000 comp rx=0 tx=0).stop 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 10 monclient: shutdown 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 msgr2=0x7fa93c154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 0x7fa93c154d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa93800be20 tx=0x7fa93800bf20 comp rx=0 tx=0).stop 2026-02-24T18:39:08.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 shutdown_connections 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa90c078e00 0x7fa90c07b2c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa93c15a5e0 0x7fa93c154d30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa93c0c99a0 0x7fa93c1f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 --2- 10.20.193.18:0/4263646820 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa93c06ff20 0x7fa93c1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 >> 10.20.193.18:0/4263646820 conn(0x7fa93c0cd780 msgr2=0x7fa93c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 shutdown_connections 2026-02-24T18:39:08.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 -- 10.20.193.18:0/4263646820 wait complete. 2026-02-24T18:39:08.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:08.367+0000 7fa943ded640 1 librados: shutdown 2026-02-24T18:39:11.380 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:11.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 10 monclient: get_monmap_and_config 2026-02-24T18:39:11.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 10 monclient: build_initial_monmap 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 10 monclient: monmap: 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:11.449363+0000 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:11.449363+0000 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:11.451 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:11.452 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:11.452 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 1 Processor -- start 2026-02-24T18:39:11.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 1 -- start start 2026-02-24T18:39:11.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.448+0000 7f7964016640 10 monclient: init 2026-02-24T18:39:11.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:11.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient: _add_conns ranks=[2,3,5,0,1,4] 2026-02-24T18:39:11.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f795c1527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:11.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f795c152bb0 0x7f795c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): picked mon.noname-a con 0x7f795c152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f795c0c5fc0 0x7f795c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): picked mon.noname-c con 0x7f795c0c5fc0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795c06e300 con 0x7f795c1527d0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f795c06f1f0 con 0x7f795c152bb0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f795c0c6470 con 0x7f795c0c5fc0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7964016640 10 monclient(hunting): authenticate will time out at 1695.130474s 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f796158a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f795c152bb0 0x7f795c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:11.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7961d8b640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f795c1527d0 0x7f795c06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:40084/0 (socket says 10.20.193.18:40084) 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.449+0000 7f7961d8b640 1 -- 10.20.193.18:0/3497771647 learned_addr learned my addr 10.20.193.18:0/3497771647 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): get_auth_request con 0x7f795c152bb0 auth_method 0 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.450+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 10 monclient(hunting): handle_auth_done global_id 5010 payload 274 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 1 -- 10.20.193.18:0/3497771647 >> v2:10.20.193.110:3300/0 conn(0x7f795c0c5fc0 msgr2=0x7f795c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 1 --2- 10.20.193.18:0/3497771647 >> v2:10.20.193.110:3300/0 conn(0x7f795c0c5fc0 0x7f795c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 1 -- 10.20.193.18:0/3497771647 >> v1:10.20.193.110:6789/0 conn(0x7f795c1527d0 legacy=0x7f795c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 10 monclient: _finish_hunting 0 2026-02-24T18:39:11.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 1 monclient: found mon.noname-a 2026-02-24T18:39:11.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:11.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.451+0000 7f796158a640 1 -- 10.20.193.18:0/3497771647 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c06e5e0 con 0x7f795c152bb0 2026-02-24T18:39:11.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.452+0000 7f796158a640 1 --2- 10.20.193.18:0/3497771647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c152bb0 0x7f795c0c5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f795c158940 tx=0x7f795002ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1afa7f49fbc63cf server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:11.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 1 -- 10.20.193.18:0/3497771647 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795003c070 con 0x7f795c152bb0 2026-02-24T18:39:11.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:11.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:11.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: dump: 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: _finish_auth 0 2026-02-24T18:39:11.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: _check_auth_tickets 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 1 -- 10.20.193.18:0/3497771647 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795002faf0 con 0x7f795c152bb0 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 5 monclient: authenticate success, global_id 5010 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 1 -- 10.20.193.18:0/3497771647 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950037440 con 0x7f795c152bb0 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 4 set_mon_vals no callback set 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7960d89640 10 monclient: dump: 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:11.465 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 10 monclient: shutdown 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 -- 10.20.193.18:0/3497771647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c152bb0 msgr2=0x7f795c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 --2- 10.20.193.18:0/3497771647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c152bb0 0x7f795c0c5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f795c158940 tx=0x7f795002ede0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 -- 10.20.193.18:0/3497771647 shutdown_connections 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 --2- 10.20.193.18:0/3497771647 >> v2:10.20.193.110:3300/0 conn(0x7f795c0c5fc0 0x7f795c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 --2- 10.20.193.18:0/3497771647 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c152bb0 0x7f795c0c5a80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 -- 10.20.193.18:0/3497771647 >> 10.20.193.18:0/3497771647 conn(0x7f795c0cd780 msgr2=0x7f795c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 -- 10.20.193.18:0/3497771647 shutdown_connections 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.454+0000 7f7964016640 1 -- 10.20.193.18:0/3497771647 wait complete. 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient: build_initial_monmap 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient: monmap: 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:11.467 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 librados: starting msgr at 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 librados: starting objecter 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 Processor -- start 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 -- start start 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 librados: setting wanted keys 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 librados: calling monclient init 2026-02-24T18:39:11.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient: init 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): picked mon.noname-a con 0x7f795c0c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f795c1527d0 0x7f795c0c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): picked mon.noname-b con 0x7f795c1527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f795c152bb0 0x7f795c0c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): picked mon.noname-c con 0x7f795c152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f795c06ee80 con 0x7f795c0c5fc0 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f795c06eba0 con 0x7f795c1527d0 2026-02-24T18:39:11.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0c93a0 con 0x7f795c152bb0 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.455+0000 7f7964016640 10 monclient(hunting): authenticate will time out at 1695.136970s 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f795c152bb0 0x7f795c0c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f795c1527d0 0x7f795c0c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:43622/0 (socket says 10.20.193.18:43622) 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 1 -- 10.20.193.18:0/2065894725 learned_addr learned my addr 10.20.193.18:0/2065894725 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): get_auth_request con 0x7f795c0c5fc0 auth_method 0 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:11.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): get_auth_request con 0x7f795c152bb0 auth_method 0 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): get_auth_request con 0x7f795c1527d0 auth_method 0 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f7961d8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:11.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796258c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.456+0000 7f796158a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 10 monclient(hunting): handle_auth_done global_id 5013 payload 824 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 -- 10.20.193.18:0/2065894725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f795c152bb0 msgr2=0x7f795c0c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f795c152bb0 0x7f795c0c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 -- 10.20.193.18:0/2065894725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f795c1527d0 msgr2=0x7f795c0c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f795c1527d0 0x7f795c0c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 10 monclient: _finish_hunting 0 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 monclient: found mon.noname-a 2026-02-24T18:39:11.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.458+0000 7f7961d8b640 1 -- 10.20.193.18:0/2065894725 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c147d90 con 0x7f795c0c5fc0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.459+0000 7f7961d8b640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c760 tx=0x7f794c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.459+0000 7f794a7fc640 10 client.?.objecter ms_handle_connect 0x7f795c0c5fc0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.459+0000 7f794a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c017070 con 0x7f795c0c5fc0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: dump: 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.473 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c005b90 con 0x7f795c0c5fc0 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 5 monclient: authenticate success, global_id 5013 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:11.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c007070 con 0x7f795c0c5fc0 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794bfff640 4 set_mon_vals no callback set 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f794a7fc640 10 monclient: dump: 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:11.475 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 10 monclient: _renew_subs 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c148070 con 0x7f795c0c5fc0 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 10 client.5013.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 10 monclient: _renew_subs 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c148630 con 0x7f795c0c5fc0 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.460+0000 7f7964016640 1 librados: init done 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.461+0000 7f7964016640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.461+0000 7f7964016640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:11.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.461+0000 7f7964016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.461+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c150a00 con 0x7f795c0c5fc0 2026-02-24T18:39:11.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.566+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f794c007210 con 0x7f795c0c5fc0 2026-02-24T18:39:11.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.566+0000 7f794a7fc640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 0x7f792c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:11.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.566+0000 7f796158a640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 0x7f792c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:11.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.566+0000 7f796158a640 10 monclient: get_auth_request con 0x7f792c078e50 auth_method 0 2026-02-24T18:39:11.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.567+0000 7f796158a640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 0x7f792c07b310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7950039000 tx=0x7f7950033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:11.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.567+0000 7f794a7fc640 10 client.5013.objecter ms_handle_connect 0x7f792c078e50 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f794c05d4a0 con 0x7f795c0c5fc0 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 10 client.5013.objecter ms_dispatch2 0x7f795c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 3 client.5013.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 3 client.5013.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 20 client.5013.objecter dump_active .. 0 homeless 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f795c150a00 con 0x7f795c0c5fc0 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:11.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.574+0000 7f794a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:11.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.673+0000 7f7964016640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:11.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.673+0000 7f7964016640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:11.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.673+0000 7f7964016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:11.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.673+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f795c0c9730 con 0x7f795c0c5fc0 2026-02-24T18:39:11.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.675+0000 7f794a7fc640 1 -- 10.20.193.18:0/2065894725 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f794c021020 con 0x7f795c0c5fc0 2026-02-24T18:39:11.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.675+0000 7f794a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:11.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.675+0000 7f794a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:11.678 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:11.679 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:11.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 10 librados: watch_flush enter 2026-02-24T18:39:11.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 10 librados: watch_flush exit 2026-02-24T18:39:11.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 20 client.5013.objecter shutdown clearing up homeless session... 2026-02-24T18:39:11.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 10 client.5013.objecter successfully canceled tick 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 msgr2=0x7f792c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 0x7f792c07b310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7950039000 tx=0x7f7950033000 comp rx=0 tx=0).stop 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 10 monclient: shutdown 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 msgr2=0x7f795c157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c760 tx=0x7f794c00cc30 comp rx=0 tx=0).stop 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 shutdown_connections 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f792c078e50 0x7f792c07b310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f795c1527d0 0x7f795c0c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f795c152bb0 0x7f795c0c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 --2- 10.20.193.18:0/2065894725 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f795c0c5fc0 0x7f795c157a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:11.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 >> 10.20.193.18:0/2065894725 conn(0x7f795c0cd780 msgr2=0x7f795c14fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:11.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 shutdown_connections 2026-02-24T18:39:11.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 -- 10.20.193.18:0/2065894725 wait complete. 2026-02-24T18:39:11.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:11.677+0000 7f7964016640 1 librados: shutdown 2026-02-24T18:39:14.690 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:14.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.758+0000 7f7023f55640 10 monclient: get_monmap_and_config 2026-02-24T18:39:14.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.758+0000 7f7023f55640 10 monclient: build_initial_monmap 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.758+0000 7f7023f55640 10 monclient: monmap: 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:14.760470+0000 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:14.760470+0000 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:14.762 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 Processor -- start 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 -- start start 2026-02-24T18:39:14.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient: init 2026-02-24T18:39:14.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient: _add_conns ranks=[3,0,5,2,4,1] 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f701c06ff20 0x7f701c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-a con 0x7f701c06ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f701c0c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f701c15a5e0 0x7f701c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-c con 0x7f701c15a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701c0c7e90 con 0x7f701c0c99a0 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f701c159a40 con 0x7f701c06ff20 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:14.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f701c0c8170 con 0x7f701c15a5e0 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.759+0000 7f7023f55640 10 monclient(hunting): authenticate will time out at 1698.441594s 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f701c06ff20 0x7f701c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f701c15a5e0 0x7f701c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70214c9640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f701c0c99a0 0x7f701c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:57690/0 (socket says 10.20.193.18:57690) 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70214c9640 1 -- 10.20.193.18:0/3065799045 learned_addr learned my addr 10.20.193.18:0/3065799045 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): get_auth_request con 0x7f701c15a5e0 auth_method 0 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): get_auth_request con 0x7f701c06ff20 auth_method 0 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:14.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.760+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 10 monclient(hunting): handle_auth_done global_id 4955 payload 274 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 1 -- 10.20.193.18:0/3065799045 >> v2:10.20.193.18:3300/0 conn(0x7f701c06ff20 msgr2=0x7f701c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 1 --2- 10.20.193.18:0/3065799045 >> v2:10.20.193.18:3300/0 conn(0x7f701c06ff20 0x7f701c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 1 -- 10.20.193.18:0/3065799045 >> v1:10.20.193.18:6789/0 conn(0x7f701c0c99a0 legacy=0x7f701c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 10 monclient: _finish_hunting 0 2026-02-24T18:39:14.768 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 1 monclient: found mon.noname-c 2026-02-24T18:39:14.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:14.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.761+0000 7f70224cb640 1 -- 10.20.193.18:0/3065799045 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0c8450 con 0x7f701c15a5e0 2026-02-24T18:39:14.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.763+0000 7f70224cb640 1 --2- 10.20.193.18:0/3065799045 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c14fe10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7018009d90 tx=0x7f701802f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b346e2171b9112 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:14.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 1 -- 10.20.193.18:0/3065799045 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701803c070 con 0x7f701c15a5e0 2026-02-24T18:39:14.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:14.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:14.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 10 monclient: dump: 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 10 monclient: _finish_auth 0 2026-02-24T18:39:14.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 10 monclient: _check_auth_tickets 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7020cc8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7023f55640 5 monclient: authenticate success, global_id 4955 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.764+0000 7f7023f55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 1 -- 10.20.193.18:0/3065799045 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7018037440 con 0x7f701c15a5e0 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 1 -- 10.20.193.18:0/3065799045 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018037810 con 0x7f701c15a5e0 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7020cc8640 10 monclient: dump: 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:14.777 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 4 set_mon_vals no callback set 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 10 monclient: shutdown 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:14.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 -- 10.20.193.18:0/3065799045 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 msgr2=0x7f701c14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 --2- 10.20.193.18:0/3065799045 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c14fe10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7018009d90 tx=0x7f701802f1e0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 -- 10.20.193.18:0/3065799045 shutdown_connections 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 --2- 10.20.193.18:0/3065799045 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c14fe10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 --2- 10.20.193.18:0/3065799045 >> v2:10.20.193.18:3300/0 conn(0x7f701c06ff20 0x7f701c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 -- 10.20.193.18:0/3065799045 >> 10.20.193.18:0/3065799045 conn(0x7f701c0cd780 msgr2=0x7f701c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 -- 10.20.193.18:0/3065799045 shutdown_connections 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.765+0000 7f7023f55640 1 -- 10.20.193.18:0/3065799045 wait complete. 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient: build_initial_monmap 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient: monmap: 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:14.779 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 librados: starting msgr at 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 librados: starting objecter 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 Processor -- start 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 -- start start 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 librados: setting wanted keys 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 librados: calling monclient init 2026-02-24T18:39:14.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient: init 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 0x7f701c0c0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-a con 0x7f701c06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f701c0c99a0 0x7f701c0c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-b con 0x7f701c0c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c0c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): picked mon.noname-c con 0x7f701c15a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f701c150350 con 0x7f701c06ff20 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0c8a10 con 0x7f701c0c99a0 2026-02-24T18:39:14.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1504d0 con 0x7f701c15a5e0 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7023f55640 10 monclient(hunting): authenticate will time out at 1698.448732s 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.766+0000 7f7021cca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 0x7f701c0c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c0c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 0x7f701c0c0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:43642/0 (socket says 10.20.193.18:43642) 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 1 -- 10.20.193.18:0/2872998758 learned_addr learned my addr 10.20.193.18:0/2872998758 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): get_auth_request con 0x7f701c06ff20 auth_method 0 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:14.782 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): get_auth_request con 0x7f701c15a5e0 auth_method 0 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f701c0c99a0 0x7f701c0c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70224cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f7021cca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): get_auth_request con 0x7f701c0c99a0 auth_method 0 2026-02-24T18:39:14.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.767+0000 7f70214c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 10 monclient(hunting): handle_auth_done global_id 4958 payload 824 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 -- 10.20.193.18:0/2872998758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f701c0c99a0 msgr2=0x7f701c0c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f701c0c99a0 0x7f701c0c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 -- 10.20.193.18:0/2872998758 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 msgr2=0x7f701c0c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 0x7f701c0c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 10 monclient: _finish_hunting 0 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 monclient: found mon.noname-c 2026-02-24T18:39:14.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.768+0000 7f70224cb640 1 -- 10.20.193.18:0/2872998758 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0be9e0 con 0x7f701c15a5e0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.770+0000 7f70224cb640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c0c3720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7018036000 tx=0x7f7018004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.770+0000 7f700a7fc640 10 client.?.objecter ms_handle_connect 0x7f701c15a5e0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.770+0000 7f700a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701803c050 con 0x7f701c15a5e0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: dump: 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.785 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 5 monclient: authenticate success, global_id 4958 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 monclient: _renew_subs 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c0becc0 con 0x7f701c15a5e0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 client.4958.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 monclient: _renew_subs 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c0bb760 con 0x7f701c15a5e0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 1 librados: init done 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701803d040 con 0x7f701c15a5e0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018041810 con 0x7f701c15a5e0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700bfff640 4 set_mon_vals no callback set 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f700a7fc640 10 monclient: dump: 2026-02-24T18:39:14.787 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.771+0000 7f7023f55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.772+0000 7f7023f55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:14.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.772+0000 7f7023f55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.772+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c150860 con 0x7f701c15a5e0 2026-02-24T18:39:14.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.877+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f70180419b0 con 0x7f701c15a5e0 2026-02-24T18:39:14.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.878+0000 7f700a7fc640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 0x7f6fec07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:14.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.878+0000 7f7021cca640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 0x7f6fec07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:14.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.878+0000 7f7021cca640 10 monclient: get_auth_request con 0x7f6fec078e00 auth_method 0 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f7021cca640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 0x7f6fec07b2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f700c009d00 tx=0x7f700c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 10 client.4958.objecter ms_handle_connect 0x7f6fec078e00 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7018083ff0 con 0x7f701c15a5e0 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 10 client.4958.objecter ms_dispatch2 0x7f701c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 3 client.4958.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 3 client.4958.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:14.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.879+0000 7f700a7fc640 20 client.4958.objecter dump_active .. 0 homeless 2026-02-24T18:39:14.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.886+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7018087fb0 con 0x7f701c15a5e0 2026-02-24T18:39:14.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.886+0000 7f700a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:14.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.886+0000 7f700a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:14.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.987+0000 7f7023f55640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:14.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.987+0000 7f7023f55640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:14.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.987+0000 7f7023f55640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:14.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.987+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f701c159570 con 0x7f701c15a5e0 2026-02-24T18:39:14.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f700a7fc640 1 -- 10.20.193.18:0/2872998758 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f701808c2b0 con 0x7f701c15a5e0 2026-02-24T18:39:14.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f700a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:14.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f700a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:14.992 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:14.992 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:14.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 10 librados: watch_flush enter 2026-02-24T18:39:14.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 10 librados: watch_flush exit 2026-02-24T18:39:14.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 20 client.4958.objecter shutdown clearing up homeless session... 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 10 client.4958.objecter successfully canceled tick 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 msgr2=0x7f6fec07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 0x7f6fec07b2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f700c009d00 tx=0x7f700c009290 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 10 monclient: shutdown 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 msgr2=0x7f701c0c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c0c3720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7018036000 tx=0x7f7018004b40 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 shutdown_connections 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6fec078e00 0x7f6fec07b2c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f701c0c99a0 0x7f701c0c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f701c15a5e0 0x7f701c0c3720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 --2- 10.20.193.18:0/2872998758 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f701c06ff20 0x7f701c0c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 >> 10.20.193.18:0/2872998758 conn(0x7f701c0cd780 msgr2=0x7f701c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 shutdown_connections 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 -- 10.20.193.18:0/2872998758 wait complete. 2026-02-24T18:39:14.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:14.990+0000 7f7023f55640 1 librados: shutdown 2026-02-24T18:39:18.003 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:18.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 10 monclient: get_monmap_and_config 2026-02-24T18:39:18.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 10 monclient: build_initial_monmap 2026-02-24T18:39:18.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 10 monclient: monmap: 2026-02-24T18:39:18.072 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:18.072 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:18.071292+0000 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:18.071292+0000 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 1 Processor -- start 2026-02-24T18:39:18.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 1 -- start start 2026-02-24T18:39:18.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.070+0000 7f26fce1a640 10 monclient: init 2026-02-24T18:39:18.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:18.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient: _add_conns ranks=[3,1,4,0,2,5] 2026-02-24T18:39:18.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f26f81527d0 0x7f26f806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-a con 0x7f26f81527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:18.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f26f8152bb0 addr v1:10.20.193.32:6789/0 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f26f80c6190 0x7f26f80c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-b con 0x7f26f80c6190 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f806e300 con 0x7f26f8152bb0 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f26f806f1f0 con 0x7f26f81527d0 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f26f806e5e0 con 0x7f26f80c6190 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26fce1a640 10 monclient(hunting): authenticate will time out at 1701.752419s 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.071+0000 7f26f6575640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f26f81527d0 0x7f26f806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f5d74640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f26f8152bb0 0x7f26f80c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:51432/0 (socket says 10.20.193.18:51432) 2026-02-24T18:39:18.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f5d74640 1 -- 10.20.193.18:0/198029685 learned_addr learned my addr 10.20.193.18:0/198029685 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): get_auth_request con 0x7f26f81527d0 auth_method 0 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6d76640 1 --2- 10.20.193.18:0/198029685 >> v2:10.20.193.32:3300/0 conn(0x7f26f80c6190 0x7f26f80c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6d76640 10 monclient(hunting): get_auth_request con 0x7f26f80c6190 auth_method 0 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.072+0000 7f26f6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:18.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 10 monclient(hunting): handle_auth_done global_id 5025 payload 274 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 1 -- 10.20.193.18:0/198029685 >> v2:10.20.193.32:3300/0 conn(0x7f26f80c6190 msgr2=0x7f26f80c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 1 --2- 10.20.193.18:0/198029685 >> v2:10.20.193.32:3300/0 conn(0x7f26f80c6190 0x7f26f80c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 1 -- 10.20.193.18:0/198029685 >> v1:10.20.193.32:6789/0 conn(0x7f26f8152bb0 legacy=0x7f26f80c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 10 monclient: _finish_hunting 0 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 1 monclient: found mon.noname-a 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.073+0000 7f26f6575640 1 -- 10.20.193.18:0/198029685 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f806e8c0 con 0x7f26f81527d0 2026-02-24T18:39:18.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.074+0000 7f26f6575640 1 --2- 10.20.193.18:0/198029685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f81527d0 0x7f26f806da60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26f8158940 tx=0x7f26ec02ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e560c75853ff64db server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:18.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 1 -- 10.20.193.18:0/198029685 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec03c070 con 0x7f26f81527d0 2026-02-24T18:39:18.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: dump: 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:18.086 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: _finish_auth 0 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: _check_auth_tickets 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 1 -- 10.20.193.18:0/198029685 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ec02faf0 con 0x7f26f81527d0 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 5 monclient: authenticate success, global_id 5025 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 1 -- 10.20.193.18:0/198029685 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec037440 con 0x7f26f81527d0 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 4 set_mon_vals no callback set 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:18.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26f5573640 10 monclient: dump: 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 10 monclient: shutdown 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 -- 10.20.193.18:0/198029685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f81527d0 msgr2=0x7f26f806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 --2- 10.20.193.18:0/198029685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f81527d0 0x7f26f806da60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26f8158940 tx=0x7f26ec02ede0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 -- 10.20.193.18:0/198029685 shutdown_connections 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 --2- 10.20.193.18:0/198029685 >> v2:10.20.193.32:3300/0 conn(0x7f26f80c6190 0x7f26f80c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 --2- 10.20.193.18:0/198029685 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f81527d0 0x7f26f806da60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 -- 10.20.193.18:0/198029685 >> 10.20.193.18:0/198029685 conn(0x7f26f80cd780 msgr2=0x7f26f80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 -- 10.20.193.18:0/198029685 shutdown_connections 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.076+0000 7f26fce1a640 1 -- 10.20.193.18:0/198029685 wait complete. 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient: build_initial_monmap 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient: monmap: 2026-02-24T18:39:18.089 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 librados: starting msgr at 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 librados: starting objecter 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 Processor -- start 2026-02-24T18:39:18.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 -- start start 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 librados: setting wanted keys 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 librados: calling monclient init 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient: init 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f26f80c6190 0x7f26f80c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-b con 0x7f26f80c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f26f81527d0 0x7f26f80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-c con 0x7f26f81527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): picked mon.noname-a con 0x7f26f8152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:18.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f26f806dfa0 con 0x7f26f8152bb0 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26f806ee80 con 0x7f26f80c6190 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80c93a0 con 0x7f26f81527d0 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.077+0000 7f26fce1a640 10 monclient(hunting): authenticate will time out at 1701.758948s 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f26f80c6190 0x7f26f80c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f26f81527d0 0x7f26f80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:43662/0 (socket says 10.20.193.18:43662) 2026-02-24T18:39:18.092 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 1 -- 10.20.193.18:0/917824415 learned_addr learned my addr 10.20.193.18:0/917824415 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): get_auth_request con 0x7f26f8152bb0 auth_method 0 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): get_auth_request con 0x7f26f80c6190 auth_method 0 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): get_auth_request con 0x7f26f81527d0 auth_method 0 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:18.093 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.078+0000 7f26f5d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 10 monclient(hunting): handle_auth_done global_id 5028 payload 824 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 -- 10.20.193.18:0/917824415 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f26f81527d0 msgr2=0x7f26f80cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f26f81527d0 0x7f26f80cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 -- 10.20.193.18:0/917824415 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f26f80c6190 msgr2=0x7f26f80c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.094 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f26f80c6190 0x7f26f80c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 10 monclient: _finish_hunting 0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 monclient: found mon.noname-a 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.080+0000 7f26f6d76640 1 -- 10.20.193.18:0/917824415 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f80c4200 con 0x7f26f8152bb0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.081+0000 7f26f6d76640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26e8007e70 tx=0x7f26e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.081+0000 7f26deffd640 10 client.?.objecter ms_handle_connect 0x7f26f8152bb0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.081+0000 7f26deffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800eea0 con 0x7f26f8152bb0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: dump: 2026-02-24T18:39:18.095 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: _finish_auth 0 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:18.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8019070 con 0x7f26f8152bb0 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 5 monclient: authenticate success, global_id 5028 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _renew_subs 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f80c3690 con 0x7f26f8152bb0 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8014880 con 0x7f26f8152bb0 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26deffd640 10 monclient: dump: 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:18.097 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26f4d72640 4 set_mon_vals no callback set 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 client.5028.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _renew_subs 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.098 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f80c3c40 con 0x7f26f8152bb0 2026-02-24T18:39:18.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 1 librados: init done 2026-02-24T18:39:18.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:18.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:18.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.083+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f80c9780 con 0x7f26f8152bb0 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.095+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f26e8014a20 con 0x7f26f8152bb0 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.095+0000 7f26deffd640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 0x7f26c407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.096+0000 7f26f6575640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 0x7f26c407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.096+0000 7f26f6575640 10 monclient: get_auth_request con 0x7f26c4078e50 auth_method 0 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.096+0000 7f26f6575640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 0x7f26c407b310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26ec039000 tx=0x7f26ec033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:18.100 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.096+0000 7f26deffd640 10 client.5028.objecter ms_handle_connect 0x7f26c4078e50 2026-02-24T18:39:18.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f26e805f050 con 0x7f26f8152bb0 2026-02-24T18:39:18.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 10 client.5028.objecter ms_dispatch2 0x7f26f8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:18.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 3 client.5028.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:18.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 3 client.5028.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:18.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 20 client.5028.objecter dump_active .. 0 homeless 2026-02-24T18:39:18.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f26e8063010 con 0x7f26f8152bb0 2026-02-24T18:39:18.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:18.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.197+0000 7f26deffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:18.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.297+0000 7f26fce1a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:18.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.297+0000 7f26fce1a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:18.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.297+0000 7f26fce1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:18.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.297+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f26f80cb680 con 0x7f26f8152bb0 2026-02-24T18:39:18.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.299+0000 7f26deffd640 1 -- 10.20.193.18:0/917824415 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f26e8067310 con 0x7f26f8152bb0 2026-02-24T18:39:18.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26deffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:18.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26deffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:18.302 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:18.302 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:18.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 10 librados: watch_flush enter 2026-02-24T18:39:18.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 10 librados: watch_flush exit 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 20 client.5028.objecter shutdown clearing up homeless session... 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 10 client.5028.objecter successfully canceled tick 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 msgr2=0x7f26c407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 0x7f26c407b310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26ec039000 tx=0x7f26ec033000 comp rx=0 tx=0).stop 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 10 monclient: shutdown 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 msgr2=0x7f26f8073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.300+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26e8007e70 tx=0x7f26e800e4b0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 shutdown_connections 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f26c4078e50 0x7f26c407b310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f26f8152bb0 0x7f26f8073be0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f26f81527d0 0x7f26f80cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 --2- 10.20.193.18:0/917824415 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f26f80c6190 0x7f26f80c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 >> 10.20.193.18:0/917824415 conn(0x7f26f80cd780 msgr2=0x7f26f80749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 shutdown_connections 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 -- 10.20.193.18:0/917824415 wait complete. 2026-02-24T18:39:18.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:18.301+0000 7f26fce1a640 1 librados: shutdown 2026-02-24T18:39:19.596 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:57.541 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.623 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:57.531 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.623 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:57.568 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.625 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:57.570 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.625 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:57.533 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.626 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:57.572 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.632 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:05:57.578 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.633 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:05:57.541 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:19.635 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:05:57.580 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:21.311 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.380+0000 7f487162b640 10 monclient: get_monmap_and_config 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.380+0000 7f487162b640 10 monclient: build_initial_monmap 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.380+0000 7f487162b640 10 monclient: monmap: 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:21.382 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:21.381510+0000 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:21.381510+0000 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:21.383 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 Processor -- start 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 -- start start 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient: init 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient: _add_conns ranks=[5,2,0,1,4,3] 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f486c06ff20 0x7f486c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): picked mon.noname-c con 0x7f486c06ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f486c0c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f486c15a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:21.386 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0c7e90 con 0x7f486c15a5e0 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0c8170 con 0x7f486c0c99a0 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f486c159a40 con 0x7f486c06ff20 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.381+0000 7f487162b640 10 monclient(hunting): authenticate will time out at 1705.062632s 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486b7fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f486c15a5e0 0x7f486c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37400/0 (socket says 10.20.193.18:37400) 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486b7fe640 1 -- 10.20.193.18:0/828787678 learned_addr learned my addr 10.20.193.18:0/828787678 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486affd640 1 --2- 10.20.193.18:0/828787678 >> v2:10.20.193.110:3300/0 conn(0x7f486c06ff20 0x7f486c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486affd640 10 monclient(hunting): get_auth_request con 0x7f486c06ff20 auth_method 0 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.387 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.382+0000 7f486affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525566849 0 0) 0x7f486c0c7e90 con 0x7f486c15a5e0 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 10 monclient(hunting): my global_id is 5031 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.383+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4858003620 con 0x7f486c15a5e0 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524483071 0 0) 0x7f486c0c8170 con 0x7f486c0c99a0 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.388 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 10 monclient(hunting): my global_id is 4967 2026-02-24T18:39:21.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.384+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f486c0c7e90 con 0x7f486c0c99a0 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 10 monclient(hunting): handle_auth_done global_id 4964 payload 274 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 1 -- 10.20.193.18:0/828787678 >> v1:10.20.193.110:6789/0 conn(0x7f486c0c99a0 legacy=0x7f486c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 1 -- 10.20.193.18:0/828787678 >> v1:10.20.193.18:6789/0 conn(0x7f486c15a5e0 legacy=0x7f486c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 10 monclient: _finish_hunting 0 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 1 monclient: found mon.noname-c 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.385+0000 7f486affd640 1 -- 10.20.193.18:0/828787678 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c0c8450 con 0x7f486c06ff20 2026-02-24T18:39:21.390 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.387+0000 7f486affd640 1 --2- 10.20.193.18:0/828787678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c06ff20 0x7f486c070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48600098d0 tx=0x7f486002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c587c39f940a0f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:21.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486003c070 con 0x7f486c06ff20 2026-02-24T18:39:21.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: dump: 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:21.396 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: _finish_auth 0 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 5 monclient: authenticate success, global_id 4964 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486002fba0 con 0x7f486c06ff20 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 1 -- 10.20.193.18:0/828787678 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860037440 con 0x7f486c06ff20 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f4869ffb640 10 monclient: dump: 2026-02-24T18:39:21.397 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 4 set_mon_vals no callback set 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 10 monclient: shutdown 2026-02-24T18:39:21.398 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 -- 10.20.193.18:0/828787678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c06ff20 msgr2=0x7f486c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 --2- 10.20.193.18:0/828787678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c06ff20 0x7f486c070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48600098d0 tx=0x7f486002eed0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 -- 10.20.193.18:0/828787678 shutdown_connections 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 --2- 10.20.193.18:0/828787678 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c06ff20 0x7f486c070300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 -- 10.20.193.18:0/828787678 >> 10.20.193.18:0/828787678 conn(0x7f486c0cd780 msgr2=0x7f486c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 -- 10.20.193.18:0/828787678 shutdown_connections 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 1 -- 10.20.193.18:0/828787678 wait complete. 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 10 monclient: build_initial_monmap 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.389+0000 7f487162b640 10 monclient: monmap: 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:21.399 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 librados: starting msgr at 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 librados: starting objecter 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 Processor -- start 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 -- start start 2026-02-24T18:39:21.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 librados: setting wanted keys 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 librados: calling monclient init 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient: init 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f486c06ff20 0x7f486c14ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): picked mon.noname-a con 0x7f486c06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): picked mon.noname-c con 0x7f486c0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f486c15a5e0 0x7f486c155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): picked mon.noname-b con 0x7f486c15a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f486c150520 con 0x7f486c06ff20 2026-02-24T18:39:21.401 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0c8a10 con 0x7f486c15a5e0 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f486c1506a0 con 0x7f486c0c99a0 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f487162b640 10 monclient(hunting): authenticate will time out at 1705.071658s 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f486a7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.390+0000 7f486affd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f486c06ff20 0x7f486c14ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:39672/0 (socket says 10.20.193.18:39672) 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 1 -- 10.20.193.18:0/1955091094 learned_addr learned my addr 10.20.193.18:0/1955091094 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): get_auth_request con 0x7f486c0c99a0 auth_method 0 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:21.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): get_auth_request con 0x7f486c06ff20 auth_method 0 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f486c15a5e0 0x7f486c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): get_auth_request con 0x7f486c15a5e0 auth_method 0 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:21.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.391+0000 7f486b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 10 monclient(hunting): handle_auth_done global_id 4970 payload 824 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 -- 10.20.193.18:0/1955091094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f486c15a5e0 msgr2=0x7f486c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f486c15a5e0 0x7f486c155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 -- 10.20.193.18:0/1955091094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f486c06ff20 msgr2=0x7f486c14ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f486c06ff20 0x7f486c14ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:39:21.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 monclient: found mon.noname-c 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.392+0000 7f486a7fc640 1 -- 10.20.193.18:0/1955091094 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c073bc0 con 0x7f486c0c99a0 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.393+0000 7f486a7fc640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f485400c730 tx=0x7f485400cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.393+0000 7f484b7fe640 10 client.?.objecter ms_handle_connect 0x7f486c0c99a0 2026-02-24T18:39:21.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.393+0000 7f484b7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48540052d0 con 0x7f486c0c99a0 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 10 monclient: dump: 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:21.407 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f484b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:21.408 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 5 monclient: authenticate success, global_id 4970 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 10 monclient: _renew_subs 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c154460 con 0x7f486c0c99a0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 10 client.4970.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 10 monclient: _renew_subs 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c154a20 con 0x7f486c0c99a0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.394+0000 7f487162b640 1 librados: init done 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854005470 con 0x7f486c0c99a0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854003890 con 0x7f486c0c99a0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:21.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f484b7fe640 10 monclient: dump: 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f48697fa640 4 set_mon_vals no callback set 2026-02-24T18:39:21.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f487162b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:21.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f487162b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:21.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f487162b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.395+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c070300 con 0x7f486c0c99a0 2026-02-24T18:39:21.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.499+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4854003a30 con 0x7f486c0c99a0 2026-02-24T18:39:21.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.499+0000 7f484b7fe640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 0x7f483807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:21.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.500+0000 7f486affd640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 0x7f483807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:21.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.500+0000 7f486affd640 10 monclient: get_auth_request con 0x7f4838078e30 auth_method 0 2026-02-24T18:39:21.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.500+0000 7f486affd640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 0x7f483807b2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4860039000 tx=0x7f4860033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:21.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.500+0000 7f484b7fe640 10 client.4970.objecter ms_handle_connect 0x7f4838078e30 2026-02-24T18:39:21.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.501+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f48540979f0 con 0x7f486c0c99a0 2026-02-24T18:39:21.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.501+0000 7f484b7fe640 10 client.4970.objecter ms_dispatch2 0x7f486c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:21.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.501+0000 7f484b7fe640 3 client.4970.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:21.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.501+0000 7f484b7fe640 3 client.4970.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:21.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.501+0000 7f484b7fe640 20 client.4970.objecter dump_active .. 0 homeless 2026-02-24T18:39:21.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.508+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f48540620b0 con 0x7f486c0c99a0 2026-02-24T18:39:21.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.508+0000 7f484b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:21.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.508+0000 7f484b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:21.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.608+0000 7f487162b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:21.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.608+0000 7f487162b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:21.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.608+0000 7f487162b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:21.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.608+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f486c0cc1e0 con 0x7f486c0c99a0 2026-02-24T18:39:21.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.611+0000 7f484b7fe640 1 -- 10.20.193.18:0/1955091094 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f48540663b0 con 0x7f486c0c99a0 2026-02-24T18:39:21.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.611+0000 7f484b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:21.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.611+0000 7f484b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:21.612 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:21.613 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 10 librados: watch_flush enter 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 10 librados: watch_flush exit 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 20 client.4970.objecter shutdown clearing up homeless session... 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 10 client.4970.objecter successfully canceled tick 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 msgr2=0x7f483807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 0x7f483807b2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4860039000 tx=0x7f4860033000 comp rx=0 tx=0).stop 2026-02-24T18:39:21.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 10 monclient: shutdown 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 msgr2=0x7f486c155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f485400c730 tx=0x7f485400cc00 comp rx=0 tx=0).stop 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 shutdown_connections 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4838078e30 0x7f483807b2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f486c15a5e0 0x7f486c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f486c0c99a0 0x7f486c155220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 --2- 10.20.193.18:0/1955091094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f486c06ff20 0x7f486c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 >> 10.20.193.18:0/1955091094 conn(0x7f486c0cd780 msgr2=0x7f486c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 shutdown_connections 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 -- 10.20.193.18:0/1955091094 wait complete. 2026-02-24T18:39:21.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:21.612+0000 7f487162b640 1 librados: shutdown 2026-02-24T18:39:24.623 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.691+0000 7effeffff640 10 monclient: get_monmap_and_config 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.691+0000 7effeffff640 10 monclient: build_initial_monmap 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.691+0000 7effeffff640 10 monclient: monmap: 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:24.692740+0000 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:24.692740+0000 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:24.694 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 Processor -- start 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 -- start start 2026-02-24T18:39:24.695 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient: init 2026-02-24T18:39:24.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:24.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient: _add_conns ranks=[4,1,3,5,2,0] 2026-02-24T18:39:24.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 0x7efff00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): picked mon.noname-b con 0x7efff00c8fe0 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:24.698 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efff006d930 addr v1:10.20.193.32:6789/0 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efff006dd80 0x7efff0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): picked mon.noname-a con 0x7efff006dd80 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff0159790 con 0x7efff006d930 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7efff006e290 con 0x7efff006dd80 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7efff006e570 con 0x7efff00c8fe0 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.692+0000 7effeffff640 10 monclient(hunting): authenticate will time out at 1708.373969s 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 0x7efff00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effef7fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7efff006dd80 0x7efff0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 0x7efff00c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:53322/0 (socket says 10.20.193.18:53322) 2026-02-24T18:39:24.699 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 1 -- 10.20.193.18:0/1054984714 learned_addr learned my addr 10.20.193.18:0/1054984714 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): get_auth_request con 0x7efff00c8fe0 auth_method 0 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effef7fe640 10 monclient(hunting): get_auth_request con 0x7efff006dd80 auth_method 0 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.693+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.694+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:24.700 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.694+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.694+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.694+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 10 monclient(hunting): handle_auth_done global_id 5037 payload 274 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 1 -- 10.20.193.18:0/1054984714 >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 msgr2=0x7efff00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 1 --2- 10.20.193.18:0/1054984714 >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 0x7efff00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 1 -- 10.20.193.18:0/1054984714 >> v1:10.20.193.32:6789/0 conn(0x7efff006d930 legacy=0x7efff015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 1 monclient: found mon.noname-a 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.695+0000 7effef7fe640 1 -- 10.20.193.18:0/1054984714 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff006e850 con 0x7efff006dd80 2026-02-24T18:39:24.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.696+0000 7effef7fe640 1 --2- 10.20.193.18:0/1054984714 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006dd80 0x7efff0159df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36926bdb33853b3a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 1 -- 10.20.193.18:0/1054984714 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe003c070 con 0x7efff006dd80 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: dump: 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.708 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: _finish_auth 0 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: _check_auth_tickets 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 1 -- 10.20.193.18:0/1054984714 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe0037440 con 0x7efff006dd80 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 1 -- 10.20.193.18:0/1054984714 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0037810 con 0x7efff006dd80 2026-02-24T18:39:24.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effeffff640 5 monclient: authenticate success, global_id 5037 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effeffff640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effeffff640 4 set_mon_vals no callback set 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.697+0000 7effedffb640 10 monclient: dump: 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:24.710 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 10 monclient: shutdown 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 -- 10.20.193.18:0/1054984714 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006dd80 msgr2=0x7efff0159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 --2- 10.20.193.18:0/1054984714 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006dd80 0x7efff0159df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002f270 comp rx=0 tx=0).stop 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 -- 10.20.193.18:0/1054984714 shutdown_connections 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 --2- 10.20.193.18:0/1054984714 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006dd80 0x7efff0159df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 --2- 10.20.193.18:0/1054984714 >> v2:10.20.193.32:3300/0 conn(0x7efff00c8fe0 0x7efff00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 -- 10.20.193.18:0/1054984714 >> 10.20.193.18:0/1054984714 conn(0x7efff00cd480 msgr2=0x7efff00cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 -- 10.20.193.18:0/1054984714 shutdown_connections 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 1 -- 10.20.193.18:0/1054984714 wait complete. 2026-02-24T18:39:24.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 10 monclient: build_initial_monmap 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.698+0000 7effeffff640 10 monclient: monmap: 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 librados: starting msgr at 2026-02-24T18:39:24.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 librados: starting objecter 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 Processor -- start 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 -- start start 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 librados: setting wanted keys 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 librados: calling monclient init 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient: init 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): picked mon.noname-a con 0x7efff006d930 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efff006dd80 0x7efff0154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): picked mon.noname-b con 0x7efff006dd80 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:24.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efff00c8fe0 0x7efff0154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): picked mon.noname-c con 0x7efff00c8fe0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efff006f0f0 con 0x7efff006d930 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efff006ee10 con 0x7efff006dd80 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efff015cc10 con 0x7efff00c8fe0 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeffff640 10 monclient(hunting): authenticate will time out at 1708.380556s 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effef7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efff00c8fe0 0x7efff0154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:51558/0 (socket says 10.20.193.18:51558) 2026-02-24T18:39:24.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 1 -- 10.20.193.18:0/952554444 learned_addr learned my addr 10.20.193.18:0/952554444 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 10 monclient(hunting): get_auth_request con 0x7efff006d930 auth_method 0 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effeeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.699+0000 7effee7fc640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efff006dd80 0x7efff0154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): get_auth_request con 0x7efff006dd80 auth_method 0 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): get_auth_request con 0x7efff00c8fe0 auth_method 0 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:24.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effeeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.700+0000 7effee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 10 monclient(hunting): handle_auth_done global_id 5040 payload 824 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 -- 10.20.193.18:0/952554444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efff00c8fe0 msgr2=0x7efff0154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efff00c8fe0 0x7efff0154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 -- 10.20.193.18:0/952554444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efff006dd80 msgr2=0x7efff0154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efff006dd80 0x7efff0154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 10 monclient: _finish_hunting 0 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 monclient: found mon.noname-a 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.701+0000 7effeeffd640 1 -- 10.20.193.18:0/952554444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff01f81b0 con 0x7efff006d930 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.702+0000 7effeeffd640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effe400e890 tx=0x7effe400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.702+0000 7effcf7fe640 10 client.?.objecter ms_handle_connect 0x7efff006d930 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.702+0000 7effcf7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4017070 con 0x7efff006d930 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:24.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: dump: 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: _finish_auth 0 2026-02-24T18:39:24.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe400c940 con 0x7efff006d930 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe400cd10 con 0x7efff006d930 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effcf7fe640 10 monclient: dump: 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:24.719 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 5 monclient: authenticate success, global_id 5040 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effed7fa640 4 set_mon_vals no callback set 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _renew_subs 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff014cd70 con 0x7efff006d930 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 client.5040.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _renew_subs 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff014d330 con 0x7efff006d930 2026-02-24T18:39:24.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 1 librados: init done 2026-02-24T18:39:24.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:24.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:24.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.704+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff00c93c0 con 0x7efff006d930 2026-02-24T18:39:24.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.716+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7effe4024060 con 0x7efff006d930 2026-02-24T18:39:24.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.716+0000 7effcf7fe640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 0x7effbc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:24.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.717+0000 7effee7fc640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 0x7effbc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.717+0000 7effee7fc640 10 monclient: get_auth_request con 0x7effbc078e00 auth_method 0 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.717+0000 7effee7fc640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 0x7effbc07b2c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efff015c060 tx=0x7effd80091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.717+0000 7effcf7fe640 10 client.5040.objecter ms_handle_connect 0x7effbc078e00 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.718+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7effe405e0a0 con 0x7efff006d930 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.718+0000 7effcf7fe640 10 client.5040.objecter ms_dispatch2 0x7efff0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.718+0000 7effcf7fe640 3 client.5040.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.718+0000 7effcf7fe640 3 client.5040.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:24.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.718+0000 7effcf7fe640 20 client.5040.objecter dump_active .. 0 homeless 2026-02-24T18:39:24.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.815+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7effe4062060 con 0x7efff006d930 2026-02-24T18:39:24.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.815+0000 7effcf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:24.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.815+0000 7effcf7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:24.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.915+0000 7effeffff640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:24.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.915+0000 7effeffff640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:24.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.915+0000 7effeffff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:24.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.915+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efff006f890 con 0x7efff006d930 2026-02-24T18:39:24.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.945+0000 7effcf7fe640 1 -- 10.20.193.18:0/952554444 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7effe4066360 con 0x7efff006d930 2026-02-24T18:39:24.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.945+0000 7effcf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:24.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.945+0000 7effcf7fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:24.948 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:24.948 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 10 librados: watch_flush enter 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 10 librados: watch_flush exit 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 20 client.5040.objecter shutdown clearing up homeless session... 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 10 client.5040.objecter successfully canceled tick 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 msgr2=0x7effbc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 0x7effbc07b2c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efff015c060 tx=0x7effd80091c0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 10 monclient: shutdown 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 msgr2=0x7efff014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:24.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effe400e890 tx=0x7effe400ed60 comp rx=0 tx=0).stop 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 shutdown_connections 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7effbc078e00 0x7effbc07b2c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efff006dd80 0x7efff0154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efff00c8fe0 0x7efff0154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 --2- 10.20.193.18:0/952554444 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efff006d930 0x7efff014e9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 >> 10.20.193.18:0/952554444 conn(0x7efff00cd480 msgr2=0x7efff00c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 shutdown_connections 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 -- 10.20.193.18:0/952554444 wait complete. 2026-02-24T18:39:24.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:24.947+0000 7effeffff640 1 librados: shutdown 2026-02-24T18:39:27.959 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: get_monmap_and_config 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: build_initial_monmap 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: monmap: 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:28.029888+0000 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:28.029888+0000 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:28.031 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 Processor -- start 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 -- start start 2026-02-24T18:39:28.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: init 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient: _add_conns ranks=[0,2,5,4,3,1] 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd09006ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd0900c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd09015a5e0 0x7fd090070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-c con 0x7fd09015a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0900c7e90 con 0x7fd09006ff20 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0900c8170 con 0x7fd0900c99a0 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd090159a40 con 0x7fd09015a5e0 2026-02-24T18:39:28.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.029+0000 7fd0962ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd0962ea640 10 monclient(hunting): authenticate will time out at 1711.711002s 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd08ffff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fd09006ff20 0x7fd090159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37416/0 (socket says 10.20.193.18:37416) 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd08ffff640 1 -- 10.20.193.18:0/221927582 learned_addr learned my addr 10.20.193.18:0/221927582 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd094860640 1 --2- 10.20.193.18:0/221927582 >> v2:10.20.193.110:3300/0 conn(0x7fd09015a5e0 0x7fd090070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd094860640 10 monclient(hunting): get_auth_request con 0x7fd09015a5e0 auth_method 0 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd094860640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd094860640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.030+0000 7fd094860640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 1 -- 10.20.193.18:0/221927582 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78789668 0 0) 0x7fd0900c7e90 con 0x7fd09006ff20 2026-02-24T18:39:28.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 10 monclient(hunting): _negotiate 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 10 monclient(hunting): my global_id is 5043 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.031+0000 7fd08effd640 1 -- 10.20.193.18:0/221927582 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd074003620 con 0x7fd09006ff20 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 10 monclient(hunting): handle_auth_done global_id 4976 payload 274 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 1 -- 10.20.193.18:0/221927582 >> v1:10.20.193.110:6789/0 conn(0x7fd0900c99a0 legacy=0x7fd09014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 1 -- 10.20.193.18:0/221927582 >> v1:10.20.193.18:6789/0 conn(0x7fd09006ff20 legacy=0x7fd090159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 10 monclient: _finish_hunting 0 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 1 monclient: found mon.noname-c 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.032+0000 7fd094860640 1 -- 10.20.193.18:0/221927582 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0900c8450 con 0x7fd09015a5e0 2026-02-24T18:39:28.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.034+0000 7fd094860640 1 --2- 10.20.193.18:0/221927582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09015a5e0 0x7fd090070300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd084009dc0 tx=0x7fd08402f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca31a4d059b3b5a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 1 -- 10.20.193.18:0/221927582 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08403c070 con 0x7fd09015a5e0 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: dump: 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:28.043 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: _finish_auth 0 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 1 -- 10.20.193.18:0/221927582 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd084037440 con 0x7fd09015a5e0 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 1 -- 10.20.193.18:0/221927582 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084037810 con 0x7fd09015a5e0 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 5 monclient: authenticate success, global_id 4976 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:28.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 4 set_mon_vals no callback set 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd08effd640 10 monclient: dump: 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:28.045 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 10 monclient: shutdown 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 -- 10.20.193.18:0/221927582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09015a5e0 msgr2=0x7fd090070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 --2- 10.20.193.18:0/221927582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09015a5e0 0x7fd090070300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd084009dc0 tx=0x7fd08402f280 comp rx=0 tx=0).stop 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 -- 10.20.193.18:0/221927582 shutdown_connections 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 --2- 10.20.193.18:0/221927582 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09015a5e0 0x7fd090070300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 -- 10.20.193.18:0/221927582 >> 10.20.193.18:0/221927582 conn(0x7fd0900cd780 msgr2=0x7fd0900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 -- 10.20.193.18:0/221927582 shutdown_connections 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.036+0000 7fd0962ea640 1 -- 10.20.193.18:0/221927582 wait complete. 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient: build_initial_monmap 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient: monmap: 2026-02-24T18:39:28.046 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 librados: starting msgr at 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 librados: starting objecter 2026-02-24T18:39:28.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 Processor -- start 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 -- start start 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 librados: setting wanted keys 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 librados: calling monclient init 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient: init 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-c con 0x7fd09006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd0900c99a0 0x7fd09014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-b con 0x7fd0900c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd09015a5e0 0x7fd0901f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): picked mon.noname-a con 0x7fd09015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:28.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd090150520 con 0x7fd09015a5e0 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0900c8a10 con 0x7fd0900c99a0 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901506a0 con 0x7fd09006ff20 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.037+0000 7fd0962ea640 10 monclient(hunting): authenticate will time out at 1711.718813s 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd09015a5e0 0x7fd0901f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:39712/0 (socket says 10.20.193.18:39712) 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 1 -- 10.20.193.18:0/741889752 learned_addr learned my addr 10.20.193.18:0/741889752 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:28.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): get_auth_request con 0x7fd09006ff20 auth_method 0 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): get_auth_request con 0x7fd09015a5e0 auth_method 0 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd0900c99a0 0x7fd09014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): get_auth_request con 0x7fd0900c99a0 auth_method 0 2026-02-24T18:39:28.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd094860640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.038+0000 7fd08f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 10 monclient(hunting): handle_auth_done global_id 4982 payload 824 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 -- 10.20.193.18:0/741889752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd0900c99a0 msgr2=0x7fd09014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd0900c99a0 0x7fd09014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 -- 10.20.193.18:0/741889752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd09015a5e0 msgr2=0x7fd0901f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd09015a5e0 0x7fd0901f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 monclient: found mon.noname-c 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.039+0000 7fd08ffff640 1 -- 10.20.193.18:0/741889752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901f7d30 con 0x7fd09006ff20 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08ffff640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c700 tx=0x7fd07c00cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 client.?.objecter ms_handle_connect 0x7fd09006ff20 2026-02-24T18:39:28.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:28.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c017070 con 0x7fd09006ff20 2026-02-24T18:39:28.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:28.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:28.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 monclient: dump: 2026-02-24T18:39:28.054 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 monclient: _finish_auth 0 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:39:28.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd08cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd0962ea640 5 monclient: authenticate success, global_id 4982 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd0962ea640 10 monclient: _renew_subs 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd0962ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.041+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0901f4450 con 0x7fd09006ff20 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 client.4982.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 monclient: _renew_subs 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd090154360 con 0x7fd09006ff20 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 1 librados: init done 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd090070300 con 0x7fd09006ff20 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c005bd0 con 0x7fd09006ff20 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd08cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.042+0000 7fd08e7fc640 4 set_mon_vals no callback set 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.043+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c021060 con 0x7fd09006ff20 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.043+0000 7fd08cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.043+0000 7fd08cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.043+0000 7fd08cff9640 10 monclient: dump: 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:28.057 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:28.058 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:28.058 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:28.058 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:28.058 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:28.148 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.146+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd07c024080 con 0x7fd09006ff20 2026-02-24T18:39:28.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.147+0000 7fd08cff9640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 0x7fd06007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:28.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.147+0000 7fd08f7fe640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 0x7fd06007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:28.149 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.147+0000 7fd08f7fe640 10 monclient: get_auth_request con 0x7fd060078e00 auth_method 0 2026-02-24T18:39:28.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.148+0000 7fd08f7fe640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 0x7fd06007b2c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd080002410 tx=0x7fd080007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:28.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.148+0000 7fd08cff9640 10 client.4982.objecter ms_handle_connect 0x7fd060078e00 2026-02-24T18:39:28.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.149+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd07c096dd0 con 0x7fd09006ff20 2026-02-24T18:39:28.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.149+0000 7fd08cff9640 10 client.4982.objecter ms_dispatch2 0x7fd090000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:28.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.149+0000 7fd08cff9640 3 client.4982.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:28.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.149+0000 7fd08cff9640 3 client.4982.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:28.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.149+0000 7fd08cff9640 20 client.4982.objecter dump_active .. 0 homeless 2026-02-24T18:39:28.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.155+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd07c0615a0 con 0x7fd09006ff20 2026-02-24T18:39:28.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.155+0000 7fd08cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:28.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.155+0000 7fd08cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:28.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.261+0000 7fd0962ea640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:28.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.261+0000 7fd0962ea640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:28.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.261+0000 7fd0962ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:28.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.261+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd090158660 con 0x7fd09006ff20 2026-02-24T18:39:28.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.264+0000 7fd08cff9640 1 -- 10.20.193.18:0/741889752 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd090158660 con 0x7fd09006ff20 2026-02-24T18:39:28.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.264+0000 7fd08cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:28.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.264+0000 7fd08cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:28.266 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:28.266 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:28.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 10 librados: watch_flush enter 2026-02-24T18:39:28.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 10 librados: watch_flush exit 2026-02-24T18:39:28.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 20 client.4982.objecter shutdown clearing up homeless session... 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 10 client.4982.objecter successfully canceled tick 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 msgr2=0x7fd06007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 0x7fd06007b2c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd080002410 tx=0x7fd080007ef0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 10 monclient: shutdown 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 msgr2=0x7fd0901f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c700 tx=0x7fd07c00cbd0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 shutdown_connections 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd060078e00 0x7fd06007b2c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd0900c99a0 0x7fd09014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd09015a5e0 0x7fd0901f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 --2- 10.20.193.18:0/741889752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd09006ff20 0x7fd0901f77f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:28.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 >> 10.20.193.18:0/741889752 conn(0x7fd0900cd780 msgr2=0x7fd0900cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:28.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 shutdown_connections 2026-02-24T18:39:28.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 -- 10.20.193.18:0/741889752 wait complete. 2026-02-24T18:39:28.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:28.265+0000 7fd0962ea640 1 librados: shutdown 2026-02-24T18:39:31.277 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:31.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 10 monclient: get_monmap_and_config 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 10 monclient: build_initial_monmap 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 10 monclient: monmap: 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:31.347156+0000 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:31.347156+0000 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.348 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:31.349 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 1 Processor -- start 2026-02-24T18:39:31.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 1 -- start start 2026-02-24T18:39:31.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.346+0000 7fbeee9ed640 10 monclient: init 2026-02-24T18:39:31.352 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient: _add_conns ranks=[4,2,0,5,3,1] 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fbee806ff20 0x7fbee8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-b con 0x7fbee806ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbee80c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbee815a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee80c7e90 con 0x7fbee815a5e0 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee80c8170 con 0x7fbee80c99a0 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbee8159a40 con 0x7fbee806ff20 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeee9ed640 10 monclient(hunting): authenticate will time out at 1715.028295s 2026-02-24T18:39:31.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeecf63640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fbee815a5e0 0x7fbee814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:59326/0 (socket says 10.20.193.18:59326) 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbeecf63640 1 -- 10.20.193.18:0/1900036344 learned_addr learned my addr 10.20.193.18:0/1900036344 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.347+0000 7fbee7fff640 1 --2- 10.20.193.18:0/1900036344 >> v2:10.20.193.32:3300/0 conn(0x7fbee806ff20 0x7fbee8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): get_auth_request con 0x7fbee806ff20 auth_method 0 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632035036 0 0) 0x7fbee80c7e90 con 0x7fbee815a5e0 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 10 monclient(hunting): my global_id is 5049 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.348+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbecc003620 con 0x7fbee815a5e0 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732776819 0 0) 0x7fbee80c8170 con 0x7fbee80c99a0 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 10 monclient(hunting): my global_id is 4985 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee80c7e90 con 0x7fbee80c99a0 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 10 monclient(hunting): handle_auth_done global_id 5014 payload 274 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 1 -- 10.20.193.18:0/1900036344 >> v1:10.20.193.110:6789/0 conn(0x7fbee80c99a0 legacy=0x7fbee8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 1 -- 10.20.193.18:0/1900036344 >> v1:10.20.193.18:6789/0 conn(0x7fbee815a5e0 legacy=0x7fbee814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 10 monclient: _finish_hunting 0 2026-02-24T18:39:31.355 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 1 monclient: found mon.noname-b 2026-02-24T18:39:31.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:31.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:39:31.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.349+0000 7fbee7fff640 1 -- 10.20.193.18:0/1900036344 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee80c8450 con 0x7fbee806ff20 2026-02-24T18:39:31.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.350+0000 7fbee7fff640 1 --2- 10.20.193.18:0/1900036344 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee806ff20 0x7fbee8070300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098d0 tx=0x7fbed402eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe41b83d204a9138 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:31.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed403c070 con 0x7fbee806ff20 2026-02-24T18:39:31.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 10 monclient: dump: 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:31.363 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbee6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbeee9ed640 5 monclient: authenticate success, global_id 5014 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.351+0000 7fbeee9ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed402fba0 con 0x7fbee806ff20 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 1 -- 10.20.193.18:0/1900036344 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4037440 con 0x7fbee806ff20 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:39:31.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbee6ffd640 10 monclient: dump: 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 4 set_mon_vals no callback set 2026-02-24T18:39:31.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 10 monclient: shutdown 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 -- 10.20.193.18:0/1900036344 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee806ff20 msgr2=0x7fbee8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/1900036344 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee806ff20 0x7fbee8070300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098d0 tx=0x7fbed402eed0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 -- 10.20.193.18:0/1900036344 shutdown_connections 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/1900036344 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee806ff20 0x7fbee8070300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 -- 10.20.193.18:0/1900036344 >> 10.20.193.18:0/1900036344 conn(0x7fbee80cd780 msgr2=0x7fbee80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 -- 10.20.193.18:0/1900036344 shutdown_connections 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.352+0000 7fbeee9ed640 1 -- 10.20.193.18:0/1900036344 wait complete. 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient: build_initial_monmap 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient: monmap: 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:31.366 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 librados: starting msgr at 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 librados: starting objecter 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 Processor -- start 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 -- start start 2026-02-24T18:39:31.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 librados: setting wanted keys 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 librados: calling monclient init 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient: init 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbee806ff20 0x7fbee81f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-a con 0x7fbee806ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-c con 0x7fbee80c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee815a5e0 0x7fbee81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): picked mon.noname-b con 0x7fbee815a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbee8150520 con 0x7fbee806ff20 2026-02-24T18:39:31.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbee80c8a10 con 0x7fbee815a5e0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbee81506a0 con 0x7fbee80c99a0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.353+0000 7fbeee9ed640 10 monclient(hunting): authenticate will time out at 1715.034818s 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbee806ff20 0x7fbee81f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee815a5e0 0x7fbee81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:48292/0 (socket says 10.20.193.18:48292) 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 1 -- 10.20.193.18:0/3486605107 learned_addr learned my addr 10.20.193.18:0/3486605107 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): get_auth_request con 0x7fbee80c99a0 auth_method 0 2026-02-24T18:39:31.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): get_auth_request con 0x7fbee815a5e0 auth_method 0 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): get_auth_request con 0x7fbee806ff20 auth_method 0 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:31.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbeecf63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.354+0000 7fbee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 10 monclient(hunting): handle_auth_done global_id 4988 payload 824 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 -- 10.20.193.18:0/3486605107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee815a5e0 msgr2=0x7fbee81f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee815a5e0 0x7fbee81f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 -- 10.20.193.18:0/3486605107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbee806ff20 msgr2=0x7fbee81f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbee806ff20 0x7fbee81f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:39:31.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 monclient: found mon.noname-c 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.355+0000 7fbee77fe640 1 -- 10.20.193.18:0/3486605107 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee80c2090 con 0x7fbee80c99a0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.357+0000 7fbee77fe640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c730 tx=0x7fbed800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.357+0000 7fbee4ff9640 10 client.?.objecter ms_handle_connect 0x7fbee80c99a0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.357+0000 7fbee4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80052d0 con 0x7fbee80c99a0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: dump: 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:31.372 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8005490 con 0x7fbee80c99a0 2026-02-24T18:39:31.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80038d0 con 0x7fbee80c99a0 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee4ff9640 10 monclient: dump: 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:31.374 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbee67fc640 4 set_mon_vals no callback set 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 5 monclient: authenticate success, global_id 4988 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 10 monclient: _renew_subs 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee80c2370 con 0x7fbee80c99a0 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 10 client.4988.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 10 monclient: _renew_subs 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee80c2930 con 0x7fbee80c99a0 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.358+0000 7fbeee9ed640 1 librados: init done 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.359+0000 7fbeee9ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:31.375 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.359+0000 7fbeee9ed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:31.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.359+0000 7fbeee9ed640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:31.376 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.359+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee8070300 con 0x7fbee80c99a0 2026-02-24T18:39:31.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.463+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbed8003a70 con 0x7fbee80c99a0 2026-02-24T18:39:31.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.464+0000 7fbee4ff9640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 0x7fbeb807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:31.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.464+0000 7fbee7fff640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 0x7fbeb807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:31.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.464+0000 7fbee7fff640 10 monclient: get_auth_request con 0x7fbeb8078e30 auth_method 0 2026-02-24T18:39:31.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.465+0000 7fbee7fff640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 0x7fbeb807b2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbed4039000 tx=0x7fbed4033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:31.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.465+0000 7fbee4ff9640 10 client.4988.objecter ms_handle_connect 0x7fbeb8078e30 2026-02-24T18:39:31.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.467+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fbed805e0e0 con 0x7fbee80c99a0 2026-02-24T18:39:31.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.467+0000 7fbee4ff9640 10 client.4988.objecter ms_dispatch2 0x7fbee8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:31.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.467+0000 7fbee4ff9640 3 client.4988.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:31.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.467+0000 7fbee4ff9640 3 client.4988.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:31.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.467+0000 7fbee4ff9640 20 client.4988.objecter dump_active .. 0 homeless 2026-02-24T18:39:31.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.473+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbed80620a0 con 0x7fbee80c99a0 2026-02-24T18:39:31.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.473+0000 7fbee4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:31.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.473+0000 7fbee4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:31.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.573+0000 7fbeee9ed640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:31.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.573+0000 7fbeee9ed640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:31.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.573+0000 7fbeee9ed640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:31.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.573+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbee814f630 con 0x7fbee80c99a0 2026-02-24T18:39:31.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.576+0000 7fbee4ff9640 1 -- 10.20.193.18:0/3486605107 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbed80663a0 con 0x7fbee80c99a0 2026-02-24T18:39:31.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.576+0000 7fbee4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:31.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.576+0000 7fbee4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:31.578 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:31.578 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 10 librados: watch_flush enter 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 10 librados: watch_flush exit 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 20 client.4988.objecter shutdown clearing up homeless session... 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 10 client.4988.objecter successfully canceled tick 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 msgr2=0x7fbeb807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 0x7fbeb807b2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbed4039000 tx=0x7fbed4033000 comp rx=0 tx=0).stop 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 10 monclient: shutdown 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:31.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 msgr2=0x7fbee8073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c730 tx=0x7fbed800cc00 comp rx=0 tx=0).stop 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 shutdown_connections 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbeb8078e30 0x7fbeb807b2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbee815a5e0 0x7fbee81f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbee80c99a0 0x7fbee8073740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 --2- 10.20.193.18:0/3486605107 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbee806ff20 0x7fbee81f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 >> 10.20.193.18:0/3486605107 conn(0x7fbee80cd780 msgr2=0x7fbee814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 shutdown_connections 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 -- 10.20.193.18:0/3486605107 wait complete. 2026-02-24T18:39:31.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:31.577+0000 7fbeee9ed640 1 librados: shutdown 2026-02-24T18:39:34.587 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:34.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.656+0000 7fea2815c640 10 monclient: get_monmap_and_config 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.656+0000 7fea2815c640 10 monclient: build_initial_monmap 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.656+0000 7fea2815c640 10 monclient: monmap: 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:34.657451+0000 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:34.657451+0000 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:34.659 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 Processor -- start 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 -- start start 2026-02-24T18:39:34.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient: init 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient: _add_conns ranks=[4,3,5,0,2,1] 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fea2006ff20 0x7fea20070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-b con 0x7fea2006ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fea200c99a0 0x7fea20159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.668 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-a con 0x7fea200c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fea2015a410 0x7fea2014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-c con 0x7fea2015a410 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fea20159a40 con 0x7fea200c99a0 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fea2015a860 con 0x7fea2006ff20 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fea200c7e90 con 0x7fea2015a410 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea2815c640 10 monclient(hunting): authenticate will time out at 1718.338572s 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.657+0000 7fea256d0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fea200c99a0 0x7fea20159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fea2006ff20 0x7fea20070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fea200c99a0 0x7fea20159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:56220/0 (socket says 10.20.193.18:56220) 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 1 -- 10.20.193.18:0/3985769761 learned_addr learned my addr 10.20.193.18:0/3985769761 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): get_auth_request con 0x7fea200c99a0 auth_method 0 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 1 --2- 10.20.193.18:0/3985769761 >> v2:10.20.193.110:3300/0 conn(0x7fea2015a410 0x7fea2014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): get_auth_request con 0x7fea2006ff20 auth_method 0 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): get_auth_request con 0x7fea2015a410 auth_method 0 2026-02-24T18:39:34.670 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.658+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 10 monclient(hunting): handle_auth_done global_id 5055 payload 274 2026-02-24T18:39:34.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 -- 10.20.193.18:0/3985769761 >> v2:10.20.193.110:3300/0 conn(0x7fea2015a410 msgr2=0x7fea2014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 --2- 10.20.193.18:0/3985769761 >> v2:10.20.193.110:3300/0 conn(0x7fea2015a410 0x7fea2014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 -- 10.20.193.18:0/3985769761 >> v2:10.20.193.32:3300/0 conn(0x7fea2006ff20 msgr2=0x7fea20070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 --2- 10.20.193.18:0/3985769761 >> v2:10.20.193.32:3300/0 conn(0x7fea2006ff20 0x7fea20070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 10 monclient: _finish_hunting 0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 monclient: found mon.noname-a 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.659+0000 7fea256d0640 1 -- 10.20.193.18:0/3985769761 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200c8170 con 0x7fea200c99a0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.660+0000 7fea256d0640 1 --2- 10.20.193.18:0/3985769761 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea200c99a0 0x7fea20159ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea200c5380 tx=0x7fea1402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b76a3f166549f7 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 1 -- 10.20.193.18:0/3985769761 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1403c070 con 0x7fea200c99a0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:34.672 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 10 monclient: dump: 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 10 monclient: _finish_auth 0 2026-02-24T18:39:34.673 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 10 monclient: _check_auth_tickets 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea24ecf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea2815c640 5 monclient: authenticate success, global_id 5055 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.661+0000 7fea2815c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 1 -- 10.20.193.18:0/3985769761 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1402fb50 con 0x7fea200c99a0 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 1 -- 10.20.193.18:0/3985769761 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14037440 con 0x7fea200c99a0 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea24ecf640 10 monclient: dump: 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:34.674 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 4 set_mon_vals no callback set 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 10 monclient: shutdown 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:34.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 -- 10.20.193.18:0/3985769761 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea200c99a0 msgr2=0x7fea20159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 --2- 10.20.193.18:0/3985769761 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea200c99a0 0x7fea20159ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea200c5380 tx=0x7fea1402eed0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 -- 10.20.193.18:0/3985769761 shutdown_connections 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 --2- 10.20.193.18:0/3985769761 >> v2:10.20.193.110:3300/0 conn(0x7fea2015a410 0x7fea2014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 --2- 10.20.193.18:0/3985769761 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea200c99a0 0x7fea20159ed0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 --2- 10.20.193.18:0/3985769761 >> v2:10.20.193.32:3300/0 conn(0x7fea2006ff20 0x7fea20070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 -- 10.20.193.18:0/3985769761 >> 10.20.193.18:0/3985769761 conn(0x7fea200cd780 msgr2=0x7fea200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 -- 10.20.193.18:0/3985769761 shutdown_connections 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 1 -- 10.20.193.18:0/3985769761 wait complete. 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 10 monclient: build_initial_monmap 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.662+0000 7fea2815c640 10 monclient: monmap: 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:34.676 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 librados: starting msgr at 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 librados: starting objecter 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 Processor -- start 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 -- start start 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 librados: setting wanted keys 2026-02-24T18:39:34.677 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 librados: calling monclient init 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient: init 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-a con 0x7fea2006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fea200c99a0 0x7fea201508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-b con 0x7fea200c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fea2015a410 0x7fea2014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): picked mon.noname-c con 0x7fea2015a410 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fea200c8a10 con 0x7fea2006ff20 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.678 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea200c8730 con 0x7fea200c99a0 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea20150350 con 0x7fea2015a410 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea2815c640 10 monclient(hunting): authenticate will time out at 1718.344659s 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.663+0000 7fea25ed1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fea200c99a0 0x7fea201508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:56222/0 (socket says 10.20.193.18:56222) 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 1 -- 10.20.193.18:0/882149798 learned_addr learned my addr 10.20.193.18:0/882149798 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): get_auth_request con 0x7fea2006ff20 auth_method 0 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.679 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fea2015a410 0x7fea2014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): get_auth_request con 0x7fea2015a410 auth_method 0 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): get_auth_request con 0x7fea200c99a0 auth_method 0 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea25ed1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea266d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.664+0000 7fea256d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 10 monclient(hunting): handle_auth_done global_id 5058 payload 824 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 -- 10.20.193.18:0/882149798 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fea2015a410 msgr2=0x7fea2014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fea2015a410 0x7fea2014a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 -- 10.20.193.18:0/882149798 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fea200c99a0 msgr2=0x7fea201508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fea200c99a0 0x7fea201508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 10 monclient: _finish_hunting 0 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 monclient: found mon.noname-a 2026-02-24T18:39:34.681 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:34.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.665+0000 7fea25ed1640 1 -- 10.20.193.18:0/882149798 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea20149640 con 0x7fea2006ff20 2026-02-24T18:39:34.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.666+0000 7fea25ed1640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea1000ec10 tx=0x7fea1000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:34.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.666+0000 7fea0e7fc640 10 client.?.objecter ms_handle_connect 0x7fea2006ff20 2026-02-24T18:39:34.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.666+0000 7fea0e7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:34.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10017070 con 0x7fea2006ff20 2026-02-24T18:39:34.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: dump: 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:34.685 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 5 monclient: authenticate success, global_id 5058 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1000cc40 con 0x7fea2006ff20 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 10 monclient: _renew_subs 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea20151190 con 0x7fea2006ff20 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10013470 con 0x7fea2006ff20 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:34.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0e7fc640 10 monclient: dump: 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea0ffff640 4 set_mon_vals no callback set 2026-02-24T18:39:34.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 10 client.5058.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 10 monclient: _renew_subs 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea20151750 con 0x7fea2006ff20 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.668+0000 7fea2815c640 1 librados: init done 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.669+0000 7fea2815c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.669+0000 7fea2815c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.669+0000 7fea2815c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.669+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea201506e0 con 0x7fea2006ff20 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.680+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fea10013630 con 0x7fea2006ff20 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.680+0000 7fea0e7fc640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 0x7fe9f007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.681+0000 7fea256d0640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 0x7fe9f007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:34.688 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.681+0000 7fea256d0640 10 monclient: get_auth_request con 0x7fe9f0078e80 auth_method 0 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.681+0000 7fea256d0640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 0x7fe9f007b340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea200709c0 tx=0x7fea14039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.681+0000 7fea0e7fc640 10 client.5058.objecter ms_handle_connect 0x7fe9f0078e80 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.683+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fea1005e4d0 con 0x7fea2006ff20 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.683+0000 7fea0e7fc640 10 client.5058.objecter ms_dispatch2 0x7fea20000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.683+0000 7fea0e7fc640 3 client.5058.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.683+0000 7fea0e7fc640 3 client.5058.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:34.689 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.683+0000 7fea0e7fc640 20 client.5058.objecter dump_active .. 0 homeless 2026-02-24T18:39:34.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.783+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fea10062490 con 0x7fea2006ff20 2026-02-24T18:39:34.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.783+0000 7fea0e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:34.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.783+0000 7fea0e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:34.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.885+0000 7fea2815c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:34.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.885+0000 7fea2815c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:34.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.885+0000 7fea2815c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:34.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.885+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fea2014d5a0 con 0x7fea2006ff20 2026-02-24T18:39:34.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.887+0000 7fea0e7fc640 1 -- 10.20.193.18:0/882149798 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fea10066790 con 0x7fea2006ff20 2026-02-24T18:39:34.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.887+0000 7fea0e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:34.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.887+0000 7fea0e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:34.889 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:34.889 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:34.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 10 librados: watch_flush enter 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 10 librados: watch_flush exit 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 20 client.5058.objecter shutdown clearing up homeless session... 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 10 client.5058.objecter successfully canceled tick 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 msgr2=0x7fe9f007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 0x7fe9f007b340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea200709c0 tx=0x7fea14039040 comp rx=0 tx=0).stop 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 10 monclient: shutdown 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 msgr2=0x7fea20149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea1000ec10 tx=0x7fea1000c520 comp rx=0 tx=0).stop 2026-02-24T18:39:34.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 shutdown_connections 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe9f0078e80 0x7fe9f007b340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fea200c99a0 0x7fea201508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fea2015a410 0x7fea2014a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 --2- 10.20.193.18:0/882149798 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fea2006ff20 0x7fea20149100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 >> 10.20.193.18:0/882149798 conn(0x7fea200cd780 msgr2=0x7fea2014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 shutdown_connections 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 -- 10.20.193.18:0/882149798 wait complete. 2026-02-24T18:39:34.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:34.889+0000 7fea2815c640 1 librados: shutdown 2026-02-24T18:39:37.900 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.969+0000 7f39937a0640 10 monclient: get_monmap_and_config 2026-02-24T18:39:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.969+0000 7f39937a0640 10 monclient: build_initial_monmap 2026-02-24T18:39:37.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.969+0000 7f39937a0640 10 monclient: monmap: 2026-02-24T18:39:37.971 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:37.970650+0000 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:37.970650+0000 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:37.972 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 Processor -- start 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 -- start start 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient: init 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient: _add_conns ranks=[5,0,3,1,4,2] 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-c con 0x7f398c06ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f398c0c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f398c15a5e0 0x7f398c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:37.976 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-a con 0x7f398c15a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0c7e90 con 0x7f398c0c99a0 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f398c159a40 con 0x7f398c15a5e0 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f398c0c8170 con 0x7f398c06ff20 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.970+0000 7f39937a0640 10 monclient(hunting): authenticate will time out at 1721.651877s 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f398c15a5e0 0x7f398c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:48324/0 (socket says 10.20.193.18:48324) 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 1 -- 10.20.193.18:0/3345109600 learned_addr learned my addr 10.20.193.18:0/3345109600 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:37.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): get_auth_request con 0x7f398c06ff20 auth_method 0 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): get_auth_request con 0x7f398c15a5e0 auth_method 0 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:37.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.971+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 10 monclient(hunting): handle_auth_done global_id 5061 payload 274 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 1 -- 10.20.193.18:0/3345109600 >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 msgr2=0x7f398c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 1 --2- 10.20.193.18:0/3345109600 >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 1 -- 10.20.193.18:0/3345109600 >> v1:10.20.193.18:6789/0 conn(0x7f398c0c99a0 legacy=0x7f398c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 10 monclient: _finish_hunting 0 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 1 monclient: found mon.noname-a 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991d16640 1 -- 10.20.193.18:0/3345109600 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c0c8450 con 0x7f398c15a5e0 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.972+0000 7f3991515640 1 --2- 10.20.193.18:0/3345109600 >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:39:37.979 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.973+0000 7f3991d16640 1 --2- 10.20.193.18:0/3345109600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c14fe10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3988009d90 tx=0x7f398802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0486367bfce5a33 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:37.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 1 -- 10.20.193.18:0/3345109600 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398803c070 con 0x7f398c15a5e0 2026-02-24T18:39:37.985 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: dump: 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:37.986 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: _finish_auth 0 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 1 -- 10.20.193.18:0/3345109600 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398802fe10 con 0x7f398c15a5e0 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 5 monclient: authenticate success, global_id 5061 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 1 -- 10.20.193.18:0/3345109600 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988037660 con 0x7f398c15a5e0 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f397bfff640 10 monclient: dump: 2026-02-24T18:39:37.987 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 4 set_mon_vals no callback set 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 10 monclient: shutdown 2026-02-24T18:39:37.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 -- 10.20.193.18:0/3345109600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 msgr2=0x7f398c14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 --2- 10.20.193.18:0/3345109600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c14fe10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3988009d90 tx=0x7f398802f1e0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 -- 10.20.193.18:0/3345109600 shutdown_connections 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 --2- 10.20.193.18:0/3345109600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c14fe10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 --2- 10.20.193.18:0/3345109600 >> v2:10.20.193.110:3300/0 conn(0x7f398c06ff20 0x7f398c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 -- 10.20.193.18:0/3345109600 >> 10.20.193.18:0/3345109600 conn(0x7f398c0cd780 msgr2=0x7f398c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 -- 10.20.193.18:0/3345109600 shutdown_connections 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.976+0000 7f39937a0640 1 -- 10.20.193.18:0/3345109600 wait complete. 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient: build_initial_monmap 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient: monmap: 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:37.989 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 librados: starting msgr at 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 librados: starting objecter 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 Processor -- start 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 -- start start 2026-02-24T18:39:37.990 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 librados: setting wanted keys 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 librados: calling monclient init 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient: init 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 0x7f398c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-c con 0x7f398c06ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398c0c99a0 0x7f398c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-b con 0x7f398c0c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c1e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): picked mon.noname-a con 0x7f398c15a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f398c150350 con 0x7f398c15a5e0 2026-02-24T18:39:37.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0c8a10 con 0x7f398c0c99a0 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f398c1504d0 con 0x7f398c06ff20 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.977+0000 7f39937a0640 10 monclient(hunting): authenticate will time out at 1721.658823s 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c1e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 0x7f398c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c1e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:56236/0 (socket says 10.20.193.18:56236) 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 1 -- 10.20.193.18:0/3284888705 learned_addr learned my addr 10.20.193.18:0/3284888705 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): get_auth_request con 0x7f398c15a5e0 auth_method 0 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:37.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): get_auth_request con 0x7f398c06ff20 auth_method 0 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3990d14640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398c0c99a0 0x7f398c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991d16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:37.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3991515640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3990d14640 10 monclient(hunting): get_auth_request con 0x7f398c0c99a0 auth_method 0 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3990d14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3990d14640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.978+0000 7f3990d14640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3990d14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3990d14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3990d14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 10 monclient(hunting): handle_auth_done global_id 5000 payload 824 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 -- 10.20.193.18:0/3284888705 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398c0c99a0 msgr2=0x7f398c14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398c0c99a0 0x7f398c14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 -- 10.20.193.18:0/3284888705 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 msgr2=0x7f398c1e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c1e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 10 monclient: _finish_hunting 0 2026-02-24T18:39:37.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 monclient: found mon.noname-c 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.979+0000 7f3991515640 1 -- 10.20.193.18:0/3284888705 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c1f5700 con 0x7f398c06ff20 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.980+0000 7f3991515640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 0x7f398c14ced0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f397c00ebb0 tx=0x7f397c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.980+0000 7f3979ffb640 10 client.?.objecter ms_handle_connect 0x7f398c06ff20 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.980+0000 7f3979ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c00cc40 con 0x7f398c06ff20 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: dump: 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:37.995 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: _finish_auth 0 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 5 monclient: authenticate success, global_id 5000 2026-02-24T18:39:37.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c00cde0 con 0x7f398c06ff20 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 10 monclient: _renew_subs 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c1f59e0 con 0x7f398c06ff20 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c004230 con 0x7f398c06ff20 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f397b7fe640 4 set_mon_vals no callback set 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f3979ffb640 10 monclient: dump: 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:37.997 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 10 client.5000.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 10 monclient: _renew_subs 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c1f5fa0 con 0x7f398c06ff20 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.982+0000 7f39937a0640 1 librados: init done 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.983+0000 7f39937a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:37.998 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.983+0000 7f39937a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:37.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.983+0000 7f39937a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:37.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.983+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c150860 con 0x7f398c06ff20 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.994+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f397c0043d0 con 0x7f398c06ff20 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.995+0000 7f3979ffb640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 0x7f395c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.995+0000 7f3990d14640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 0x7f395c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.995+0000 7f3990d14640 10 monclient: get_auth_request con 0x7f395c078e50 auth_method 0 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.995+0000 7f3990d14640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 0x7f395c07b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f398c1577f0 tx=0x7f39800091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.995+0000 7f3979ffb640 10 client.5000.objecter ms_handle_connect 0x7f395c078e50 2026-02-24T18:39:38.000 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.996+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f397c05e1a0 con 0x7f398c06ff20 2026-02-24T18:39:38.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.996+0000 7f3979ffb640 10 client.5000.objecter ms_dispatch2 0x7f398c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:38.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.996+0000 7f3979ffb640 3 client.5000.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:38.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.996+0000 7f3979ffb640 3 client.5000.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:38.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:37.996+0000 7f3979ffb640 20 client.5000.objecter dump_active .. 0 homeless 2026-02-24T18:39:38.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.098+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f397c017020 con 0x7f398c06ff20 2026-02-24T18:39:38.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.098+0000 7f3979ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:38.099 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.098+0000 7f3979ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:38.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.197+0000 7f39937a0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:38.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.197+0000 7f39937a0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:38.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.197+0000 7f39937a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:38.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.197+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f398c15a000 con 0x7f398c06ff20 2026-02-24T18:39:38.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.199+0000 7f3979ffb640 1 -- 10.20.193.18:0/3284888705 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f397c062160 con 0x7f398c06ff20 2026-02-24T18:39:38.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.199+0000 7f3979ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:38.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.199+0000 7f3979ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:38.202 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:38.202 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 10 librados: watch_flush enter 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 10 librados: watch_flush exit 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 20 client.5000.objecter shutdown clearing up homeless session... 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 10 client.5000.objecter successfully canceled tick 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 msgr2=0x7f395c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 0x7f395c07b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f398c1577f0 tx=0x7f39800091c0 comp rx=0 tx=0).stop 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 10 monclient: shutdown 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:38.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 msgr2=0x7f398c14ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 0x7f398c14ced0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f397c00ebb0 tx=0x7f397c00c520 comp rx=0 tx=0).stop 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 shutdown_connections 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f395c078e50 0x7f395c07b310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f398c0c99a0 0x7f398c14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f398c15a5e0 0x7f398c1e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 --2- 10.20.193.18:0/3284888705 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f398c06ff20 0x7f398c14ced0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 >> 10.20.193.18:0/3284888705 conn(0x7f398c0cd780 msgr2=0x7f398c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 shutdown_connections 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 -- 10.20.193.18:0/3284888705 wait complete. 2026-02-24T18:39:38.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:38.200+0000 7f39937a0640 1 librados: shutdown 2026-02-24T18:39:39.621 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:17.567 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.649 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:17.594 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.656 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:17.602 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.676 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:17.621 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.676 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:17.584 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.677 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:17.585 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.677 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:17.622 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.684 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:17.592 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:39.684 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:17.629 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:41.213 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:41.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.282+0000 7fadc652c640 10 monclient: get_monmap_and_config 2026-02-24T18:39:41.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.282+0000 7fadc652c640 10 monclient: build_initial_monmap 2026-02-24T18:39:41.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.282+0000 7fadc652c640 10 monclient: monmap: 2026-02-24T18:39:41.285 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:41.284502+0000 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:41.284502+0000 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:41.286 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:41.287 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 Processor -- start 2026-02-24T18:39:41.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 -- start start 2026-02-24T18:39:41.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient: init 2026-02-24T18:39:41.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:41.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient: _add_conns ranks=[2,1,3,5,0,4] 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fadc01527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fadc0152bb0 addr v1:10.20.193.32:6789/0 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fadc00c6190 0x7fadc00c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-a con 0x7fadc00c6190 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc006e300 con 0x7fadc0152bb0 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc006e5e0 con 0x7fadc01527d0 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fadc006f1f0 con 0x7fadc00c6190 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:41.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.283+0000 7fadc652c640 10 monclient(hunting): authenticate will time out at 1724.965715s 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fadc00c6190 0x7fadc00c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fadc00c6190 0x7fadc00c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:35350/0 (socket says 10.20.193.18:35350) 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 1 -- 10.20.193.18:0/4172500254 learned_addr learned my addr 10.20.193.18:0/4172500254 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): get_auth_request con 0x7fadc00c6190 auth_method 0 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:41.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.284+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_done global_id 5067 payload 274 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 1 -- 10.20.193.18:0/4172500254 >> v1:10.20.193.110:6789/0 conn(0x7fadc01527d0 legacy=0x7fadc006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 1 -- 10.20.193.18:0/4172500254 >> v1:10.20.193.32:6789/0 conn(0x7fadc0152bb0 legacy=0x7fadc00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 10 monclient: _finish_hunting 0 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 1 monclient: found mon.noname-a 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.285+0000 7fadc4aa2640 1 -- 10.20.193.18:0/4172500254 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc006e8c0 con 0x7fadc00c6190 2026-02-24T18:39:41.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.286+0000 7fadc4aa2640 1 --2- 10.20.193.18:0/4172500254 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc00c6190 0x7fadc00c8e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadc006f370 tx=0x7fadb402f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f2e6adac43cfb2 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:41.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 1 -- 10.20.193.18:0/4172500254 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb402fdf0 con 0x7fadc00c6190 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: dump: 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:41.301 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: _finish_auth 0 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 1 -- 10.20.193.18:0/4172500254 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40373f0 con 0x7fadc00c6190 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 5 monclient: authenticate success, global_id 5067 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 1 -- 10.20.193.18:0/4172500254 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40377c0 con 0x7fadc00c6190 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:41.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 4 set_mon_vals no callback set 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadbeffd640 10 monclient: dump: 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:41.303 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 10 monclient: shutdown 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 -- 10.20.193.18:0/4172500254 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc00c6190 msgr2=0x7fadc00c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 --2- 10.20.193.18:0/4172500254 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc00c6190 0x7fadc00c8e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadc006f370 tx=0x7fadb402f2f0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 -- 10.20.193.18:0/4172500254 shutdown_connections 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 --2- 10.20.193.18:0/4172500254 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc00c6190 0x7fadc00c8e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 -- 10.20.193.18:0/4172500254 >> 10.20.193.18:0/4172500254 conn(0x7fadc00cd780 msgr2=0x7fadc00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 -- 10.20.193.18:0/4172500254 shutdown_connections 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.288+0000 7fadc652c640 1 -- 10.20.193.18:0/4172500254 wait complete. 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient: build_initial_monmap 2026-02-24T18:39:41.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient: monmap: 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 librados: starting msgr at 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 librados: starting objecter 2026-02-24T18:39:41.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 Processor -- start 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 -- start start 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 librados: setting wanted keys 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 librados: calling monclient init 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient: init 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fadc00c6190 0x7fadc0146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-b con 0x7fadc00c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fadc01527d0 0x7fadc01f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-c con 0x7fadc01527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:41.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): picked mon.noname-a con 0x7fadc0152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00c93a0 con 0x7fadc0152bb0 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fadc006ee80 con 0x7fadc00c6190 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00c9520 con 0x7fadc01527d0 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.289+0000 7fadc652c640 10 monclient(hunting): authenticate will time out at 1724.971965s 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fadc01527d0 0x7fadc01f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fadc00c6190 0x7fadc0146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:41.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:35360/0 (socket says 10.20.193.18:35360) 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 1 -- 10.20.193.18:0/3635352290 learned_addr learned my addr 10.20.193.18:0/3635352290 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): get_auth_request con 0x7fadc0152bb0 auth_method 0 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): get_auth_request con 0x7fadc01527d0 auth_method 0 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): get_auth_request con 0x7fadc00c6190 auth_method 0 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:41.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.290+0000 7fadbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 10 monclient(hunting): handle_auth_done global_id 5070 payload 824 2026-02-24T18:39:41.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 -- 10.20.193.18:0/3635352290 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fadc01527d0 msgr2=0x7fadc01f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fadc01527d0 0x7fadc01f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 -- 10.20.193.18:0/3635352290 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fadc00c6190 msgr2=0x7fadc0146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fadc00c6190 0x7fadc0146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 10 monclient: _finish_hunting 0 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 monclient: found mon.noname-a 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.292+0000 7fadc4aa2640 1 -- 10.20.193.18:0/3635352290 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01f4a60 con 0x7fadc0152bb0 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.293+0000 7fadc4aa2640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadb402f820 tx=0x7fadb4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.293+0000 7fadbcff9640 10 client.?.objecter ms_handle_connect 0x7fadc0152bb0 2026-02-24T18:39:41.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.293+0000 7fadbcff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4046020 con 0x7fadc0152bb0 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: dump: 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:41.311 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: _finish_auth 0 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: _check_auth_tickets 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40079b0 con 0x7fadc0152bb0 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4009510 con 0x7fadc0152bb0 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbcff9640 10 monclient: dump: 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:41.312 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadbe7fc640 4 set_mon_vals no callback set 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 5 monclient: authenticate success, global_id 5070 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 10 monclient: _renew_subs 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.313 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc0073ae0 con 0x7fadc0152bb0 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 10 client.5070.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 10 monclient: _renew_subs 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc00740a0 con 0x7fadc0152bb0 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.294+0000 7fadc652c640 1 librados: init done 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.295+0000 7fadc652c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.295+0000 7fadc652c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.295+0000 7fadc652c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.314 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.295+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc00c8e60 con 0x7fadc0152bb0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.306+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fadb4041450 con 0x7fadc0152bb0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.307+0000 7fadbcff9640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 0x7fad9007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.307+0000 7fadbffff640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 0x7fad9007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.307+0000 7fadbffff640 10 monclient: get_auth_request con 0x7fad90078e80 auth_method 0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.308+0000 7fadbcff9640 10 client.5070.objecter ms_handle_connect 0x7fad90078e80 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.308+0000 7fadbffff640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 0x7fad9007b340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadac0051f0 tx=0x7fadac007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.309+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fadb4007da0 con 0x7fadc0152bb0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.309+0000 7fadbcff9640 10 client.5070.objecter ms_dispatch2 0x7fadc0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.309+0000 7fadbcff9640 3 client.5070.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.309+0000 7fadbcff9640 3 client.5070.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:41.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.309+0000 7fadbcff9640 20 client.5070.objecter dump_active .. 0 homeless 2026-02-24T18:39:41.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.412+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fadb4050030 con 0x7fadc0152bb0 2026-02-24T18:39:41.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.412+0000 7fadbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:41.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.412+0000 7fadbcff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:41.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.511+0000 7fadc652c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:41.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.511+0000 7fadc652c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:41.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.511+0000 7fadc652c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:41.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.511+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fadc00c2060 con 0x7fadc0152bb0 2026-02-24T18:39:41.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.513+0000 7fadbcff9640 1 -- 10.20.193.18:0/3635352290 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fadb4087da0 con 0x7fadc0152bb0 2026-02-24T18:39:41.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.513+0000 7fadbcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:41.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.513+0000 7fadbcff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:41.517 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:41.517 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:41.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 10 librados: watch_flush enter 2026-02-24T18:39:41.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 10 librados: watch_flush exit 2026-02-24T18:39:41.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 20 client.5070.objecter shutdown clearing up homeless session... 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 10 client.5070.objecter successfully canceled tick 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 msgr2=0x7fad9007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 0x7fad9007b340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadac0051f0 tx=0x7fadac007f30 comp rx=0 tx=0).stop 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 10 monclient: shutdown 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 msgr2=0x7fadc00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadb402f820 tx=0x7fadb4004b40 comp rx=0 tx=0).stop 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 shutdown_connections 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fad90078e80 0x7fad9007b340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fadc0152bb0 0x7fadc00730a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fadc01527d0 0x7fadc01f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 --2- 10.20.193.18:0/3635352290 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fadc00c6190 0x7fadc0146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:41.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 >> 10.20.193.18:0/3635352290 conn(0x7fadc00cd780 msgr2=0x7fadc014e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:41.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 shutdown_connections 2026-02-24T18:39:41.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 -- 10.20.193.18:0/3635352290 wait complete. 2026-02-24T18:39:41.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:41.514+0000 7fadc652c640 1 librados: shutdown 2026-02-24T18:39:44.527 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:44.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.595+0000 7fccc0e32640 10 monclient: get_monmap_and_config 2026-02-24T18:39:44.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.595+0000 7fccc0e32640 10 monclient: build_initial_monmap 2026-02-24T18:39:44.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.595+0000 7fccc0e32640 10 monclient: monmap: 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:44.596474+0000 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:44.596474+0000 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:44.598 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:44.599 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:44.599 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 Processor -- start 2026-02-24T18:39:44.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 -- start start 2026-02-24T18:39:44.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient: init 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient: _add_conns ranks=[5,2,1,0,4,3] 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fccbc06ff20 0x7fccbc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-c con 0x7fccbc06ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fccbc0c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fccbc15a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc0c7e90 con 0x7fccbc15a5e0 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc0c8170 con 0x7fccbc0c99a0 2026-02-24T18:39:44.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fccbc159a40 con 0x7fccbc06ff20 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.596+0000 7fccc0e32640 10 monclient(hunting): authenticate will time out at 1728.277608s 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccbad76640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fccbc15a5e0 0x7fccbc14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:59858/0 (socket says 10.20.193.18:59858) 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccbad76640 1 -- 10.20.193.18:0/3850235077 learned_addr learned my addr 10.20.193.18:0/3850235077 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccba575640 1 --2- 10.20.193.18:0/3850235077 >> v2:10.20.193.110:3300/0 conn(0x7fccbc06ff20 0x7fccbc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccba575640 10 monclient(hunting): get_auth_request con 0x7fccbc06ff20 auth_method 0 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccba575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.597+0000 7fccba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:44.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 1 -- 10.20.193.18:0/3850235077 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309190855 0 0) 0x7fccbc0c7e90 con 0x7fccbc15a5e0 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 10 monclient(hunting): _negotiate 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 10 monclient(hunting): my global_id is 5035 2026-02-24T18:39:44.603 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.598+0000 7fccb9573640 1 -- 10.20.193.18:0/3850235077 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcca8003620 con 0x7fccbc15a5e0 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 10 monclient(hunting): handle_auth_done global_id 5009 payload 274 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 1 -- 10.20.193.18:0/3850235077 >> v1:10.20.193.110:6789/0 conn(0x7fccbc0c99a0 legacy=0x7fccbc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 1 -- 10.20.193.18:0/3850235077 >> v1:10.20.193.32:6789/0 conn(0x7fccbc15a5e0 legacy=0x7fccbc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 10 monclient: _finish_hunting 0 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 1 monclient: found mon.noname-c 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.600+0000 7fccba575640 1 -- 10.20.193.18:0/3850235077 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0c8450 con 0x7fccbc06ff20 2026-02-24T18:39:44.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.601+0000 7fccba575640 1 --2- 10.20.193.18:0/3850235077 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc06ff20 0x7fccbc070300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fccb00098d0 tx=0x7fccb002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2a97ebd00f02f2 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:44.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 1 -- 10.20.193.18:0/3850235077 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb003c070 con 0x7fccbc06ff20 2026-02-24T18:39:44.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: dump: 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:44.610 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: _finish_auth 0 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: _check_auth_tickets 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 1 -- 10.20.193.18:0/3850235077 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb002fba0 con 0x7fccbc06ff20 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 5 monclient: authenticate success, global_id 5009 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 1 -- 10.20.193.18:0/3850235077 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0037440 con 0x7fccbc06ff20 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccb9573640 10 monclient: dump: 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:44.611 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:44.612 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 4 set_mon_vals no callback set 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 10 monclient: shutdown 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 -- 10.20.193.18:0/3850235077 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc06ff20 msgr2=0x7fccbc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3850235077 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc06ff20 0x7fccbc070300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fccb00098d0 tx=0x7fccb002eed0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 -- 10.20.193.18:0/3850235077 shutdown_connections 2026-02-24T18:39:44.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3850235077 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc06ff20 0x7fccbc070300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 -- 10.20.193.18:0/3850235077 >> 10.20.193.18:0/3850235077 conn(0x7fccbc0cd780 msgr2=0x7fccbc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 -- 10.20.193.18:0/3850235077 shutdown_connections 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.604+0000 7fccc0e32640 1 -- 10.20.193.18:0/3850235077 wait complete. 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient: build_initial_monmap 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient: monmap: 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:44.614 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 librados: starting msgr at 2026-02-24T18:39:44.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 librados: starting objecter 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 Processor -- start 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 -- start start 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 librados: setting wanted keys 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 librados: calling monclient init 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient: init 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:44.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fccbc06ff20 0x7fccbc0bee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-a con 0x7fccbc06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fccbc0c99a0 0x7fccbc1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-b con 0x7fccbc0c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): picked mon.noname-c con 0x7fccbc15a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc150520 con 0x7fccbc06ff20 2026-02-24T18:39:44.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0c8a10 con 0x7fccbc0c99a0 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc1506a0 con 0x7fccbc15a5e0 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.605+0000 7fccc0e32640 10 monclient(hunting): authenticate will time out at 1728.286773s 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fccbc06ff20 0x7fccbc0bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:44.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fccbc0c99a0 0x7fccbc1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57824/0 (socket says 10.20.193.18:57824) 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): get_auth_request con 0x7fccbc06ff20 auth_method 0 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 1 -- 10.20.193.18:0/3995999523 learned_addr learned my addr 10.20.193.18:0/3995999523 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): get_auth_request con 0x7fccbc15a5e0 auth_method 0 2026-02-24T18:39:44.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): get_auth_request con 0x7fccbc0c99a0 auth_method 0 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:44.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccbad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccba575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.606+0000 7fccb9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 10 monclient(hunting): handle_auth_done global_id 5015 payload 824 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 -- 10.20.193.18:0/3995999523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fccbc0c99a0 msgr2=0x7fccbc1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fccbc0c99a0 0x7fccbc1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 -- 10.20.193.18:0/3995999523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fccbc06ff20 msgr2=0x7fccbc0bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fccbc06ff20 0x7fccbc0bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 10 monclient: _finish_hunting 0 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 monclient: found mon.noname-c 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.607+0000 7fccbad76640 1 -- 10.20.193.18:0/3995999523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0bf480 con 0x7fccbc15a5e0 2026-02-24T18:39:44.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.608+0000 7fccbad76640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fccac00caa0 tx=0x7fccac00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.608+0000 7fcca2ffd640 10 client.?.objecter ms_handle_connect 0x7fccbc15a5e0 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.608+0000 7fcca2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac00ac40 con 0x7fccbc15a5e0 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: dump: 2026-02-24T18:39:44.626 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:44.627 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccac00ade0 con 0x7fccbc15a5e0 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 5 monclient: authenticate success, global_id 5015 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac012600 con 0x7fccbc15a5e0 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fcca2ffd640 10 monclient: dump: 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:44.629 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccb8d72640 4 set_mon_vals no callback set 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 10 monclient: _renew_subs 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc0bf760 con 0x7fccbc15a5e0 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 10 client.5015.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 10 monclient: _renew_subs 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc0bfd20 con 0x7fccbc15a5e0 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.610+0000 7fccc0e32640 1 librados: init done 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.611+0000 7fccc0e32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.611+0000 7fccc0e32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:44.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.611+0000 7fccc0e32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.611+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc070300 con 0x7fccbc15a5e0 2026-02-24T18:39:44.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.712+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fccac0127a0 con 0x7fccbc15a5e0 2026-02-24T18:39:44.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fcca2ffd640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 0x7fcc8807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:44.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fccba575640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 0x7fcc8807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:44.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fccba575640 10 monclient: get_auth_request con 0x7fcc88078e00 auth_method 0 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fccac05e800 con 0x7fccbc15a5e0 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fcca2ffd640 10 client.5015.objecter ms_dispatch2 0x7fccbc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fcca2ffd640 3 client.5015.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.713+0000 7fcca2ffd640 3 client.5015.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.714+0000 7fcca2ffd640 20 client.5015.objecter dump_active .. 0 homeless 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.714+0000 7fccba575640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 0x7fcc8807b2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccb0039000 tx=0x7fccb0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:44.716 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.714+0000 7fcca2ffd640 10 client.5015.objecter ms_handle_connect 0x7fcc88078e00 2026-02-24T18:39:44.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.720+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fccac0627c0 con 0x7fccbc15a5e0 2026-02-24T18:39:44.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.720+0000 7fcca2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:44.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.720+0000 7fcca2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:44.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.818+0000 7fccc0e32640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:44.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.818+0000 7fccc0e32640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:44.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.818+0000 7fccc0e32640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:44.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.818+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fccbc076f20 con 0x7fccbc15a5e0 2026-02-24T18:39:44.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.820+0000 7fcca2ffd640 1 -- 10.20.193.18:0/3995999523 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fccac066ac0 con 0x7fccbc15a5e0 2026-02-24T18:39:44.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.820+0000 7fcca2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:44.822 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.820+0000 7fcca2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:44.822 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:44.822 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:44.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 10 librados: watch_flush enter 2026-02-24T18:39:44.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 10 librados: watch_flush exit 2026-02-24T18:39:44.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 20 client.5015.objecter shutdown clearing up homeless session... 2026-02-24T18:39:44.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 10 client.5015.objecter successfully canceled tick 2026-02-24T18:39:44.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 msgr2=0x7fcc8807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 0x7fcc8807b2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccb0039000 tx=0x7fccb0033000 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 10 monclient: shutdown 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 msgr2=0x7fccbc06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.821+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fccac00caa0 tx=0x7fccac00cf70 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 shutdown_connections 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fcc88078e00 0x7fcc8807b2c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fccbc0c99a0 0x7fccbc1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fccbc15a5e0 0x7fccbc06e4b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 --2- 10.20.193.18:0/3995999523 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fccbc06ff20 0x7fccbc0bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 >> 10.20.193.18:0/3995999523 conn(0x7fccbc0cd780 msgr2=0x7fccbc0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:44.825 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 shutdown_connections 2026-02-24T18:39:44.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 -- 10.20.193.18:0/3995999523 wait complete. 2026-02-24T18:39:44.826 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:44.822+0000 7fccc0e32640 1 librados: shutdown 2026-02-24T18:39:47.834 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.902+0000 7ff323888640 10 monclient: get_monmap_and_config 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.902+0000 7ff323888640 10 monclient: build_initial_monmap 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.902+0000 7ff323888640 10 monclient: monmap: 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:47.903260+0000 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:47.903260+0000 2026-02-24T18:39:47.904 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:47.905 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.902+0000 7ff323888640 1 Processor -- start 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 -- start start 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient: init 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient: _add_conns ranks=[5,2,4,3,0,1] 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff31c155c70 0x7ff31c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): picked mon.noname-c con 0x7ff31c155c70 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff31c0ccb30 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff31c156050 0x7ff31c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): picked mon.noname-b con 0x7ff31c156050 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31c072e90 con 0x7ff31c0ccb30 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c06f440 con 0x7ff31c156050 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c0cbcb0 con 0x7ff31c155c70 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.903+0000 7ff323888640 10 monclient(hunting): authenticate will time out at 1731.584683s 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff31c156050 0x7ff31c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff31c155c70 0x7ff31c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff320dfc640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7ff31c0ccb30 0x7ff31c15be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:38404/0 (socket says 10.20.193.18:38404) 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff320dfc640 1 -- 10.20.193.18:0/417098905 learned_addr learned my addr 10.20.193.18:0/417098905 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): get_auth_request con 0x7ff31c156050 auth_method 0 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): get_auth_request con 0x7ff31c155c70 auth_method 0 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.904+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 10 monclient(hunting): handle_auth_done global_id 5041 payload 274 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 1 -- 10.20.193.18:0/417098905 >> v2:10.20.193.110:3300/0 conn(0x7ff31c155c70 msgr2=0x7ff31c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 1 --2- 10.20.193.18:0/417098905 >> v2:10.20.193.110:3300/0 conn(0x7ff31c155c70 0x7ff31c15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 1 -- 10.20.193.18:0/417098905 >> v1:10.20.193.110:6789/0 conn(0x7ff31c0ccb30 legacy=0x7ff31c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 10 monclient: _finish_hunting 0 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 1 monclient: found mon.noname-b 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.905+0000 7ff321dfe640 1 -- 10.20.193.18:0/417098905 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c06f690 con 0x7ff31c156050 2026-02-24T18:39:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.906+0000 7ff321dfe640 1 --2- 10.20.193.18:0/417098905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c156050 0x7ff31c15d800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff31c076d30 tx=0x7ff31802f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb3b965e3d4813b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:47.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 1 -- 10.20.193.18:0/417098905 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31803c070 con 0x7ff31c156050 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: dump: 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:47.918 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: _finish_auth 0 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 1 -- 10.20.193.18:0/417098905 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff318037440 con 0x7ff31c156050 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 5 monclient: authenticate success, global_id 5041 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 1 -- 10.20.193.18:0/417098905 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318037810 con 0x7ff31c156050 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff30bfff640 10 monclient: dump: 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:47.920 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 4 set_mon_vals no callback set 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 10 monclient: shutdown 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 -- 10.20.193.18:0/417098905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c156050 msgr2=0x7ff31c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 --2- 10.20.193.18:0/417098905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c156050 0x7ff31c15d800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff31c076d30 tx=0x7ff31802f1c0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 -- 10.20.193.18:0/417098905 shutdown_connections 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 --2- 10.20.193.18:0/417098905 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c156050 0x7ff31c15d800 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 --2- 10.20.193.18:0/417098905 >> v2:10.20.193.110:3300/0 conn(0x7ff31c155c70 0x7ff31c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 -- 10.20.193.18:0/417098905 >> 10.20.193.18:0/417098905 conn(0x7ff31c0cc010 msgr2=0x7ff31c0cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 -- 10.20.193.18:0/417098905 shutdown_connections 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 1 -- 10.20.193.18:0/417098905 wait complete. 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 10 monclient: build_initial_monmap 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.908+0000 7ff323888640 10 monclient: monmap: 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:47.922 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 librados: starting msgr at 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 librados: starting objecter 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 Processor -- start 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 -- start start 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 librados: setting wanted keys 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 librados: calling monclient init 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient: init 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 0x7ff31c1ea710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): picked mon.noname-b con 0x7ff31c0ccb30 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff31c155c70 0x7ff31c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): picked mon.noname-a con 0x7ff31c155c70 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 0x7ff31c1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): picked mon.noname-c con 0x7ff31c156050 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c06ff30 con 0x7ff31c155c70 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c06fc50 con 0x7ff31c0ccb30 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c070210 con 0x7ff31c156050 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.909+0000 7ff323888640 10 monclient(hunting): authenticate will time out at 1731.590994s 2026-02-24T18:39:47.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 0x7ff31c1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff31c155c70 0x7ff31c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 0x7ff31c1ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 0x7ff31c1f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:57828/0 (socket says 10.20.193.18:57828) 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 1 -- 10.20.193.18:0/2746537841 learned_addr learned my addr 10.20.193.18:0/2746537841 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): get_auth_request con 0x7ff31c156050 auth_method 0 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): get_auth_request con 0x7ff31c155c70 auth_method 0 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:47.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): get_auth_request con 0x7ff31c0ccb30 auth_method 0 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff321dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff3215fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:47.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.910+0000 7ff320dfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 10 monclient(hunting): handle_auth_done global_id 5044 payload 824 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 -- 10.20.193.18:0/2746537841 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 msgr2=0x7ff31c1f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 0x7ff31c1f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 -- 10.20.193.18:0/2746537841 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff31c155c70 msgr2=0x7ff31c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff31c155c70 0x7ff31c1f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 10 monclient: _finish_hunting 0 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 monclient: found mon.noname-b 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.911+0000 7ff3215fd640 1 -- 10.20.193.18:0/2746537841 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c1f7050 con 0x7ff31c0ccb30 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.913+0000 7ff3215fd640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 0x7ff31c1ea710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00e870 tx=0x7ff30c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:47.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.913+0000 7ff309ffb640 10 client.?.objecter ms_handle_connect 0x7ff31c0ccb30 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.913+0000 7ff309ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff31c0ccb30 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: dump: 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:47.928 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: _finish_auth 0 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 5 monclient: authenticate success, global_id 5044 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _renew_subs 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1f7330 con 0x7ff31c0ccb30 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 client.5044.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _renew_subs 2026-02-24T18:39:47.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1f78f0 con 0x7ff31c0ccb30 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 1 librados: init done 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c00c940 con 0x7ff31c0ccb30 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c021060 con 0x7ff31c0ccb30 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff309ffb640 10 monclient: dump: 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:47.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff30b7fe640 4 set_mon_vals no callback set 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:47.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.914+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c160510 con 0x7ff31c0ccb30 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.925+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff30c026480 con 0x7ff31c0ccb30 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.926+0000 7ff309ffb640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 0x7ff2ec07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.926+0000 7ff320dfc640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 0x7ff2ec07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.926+0000 7ff320dfc640 10 monclient: get_auth_request con 0x7ff2ec078e00 auth_method 0 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff320dfc640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 0x7ff2ec07b2c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c9620 tx=0x7ff310009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 10 client.5044.objecter ms_handle_connect 0x7ff2ec078e00 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7ff30c00ccf0 con 0x7ff31c0ccb30 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 10 client.5044.objecter ms_dispatch2 0x7ff31c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 3 client.5044.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:47.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 3 client.5044.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:47.927+0000 7ff309ffb640 20 client.5044.objecter dump_active .. 0 homeless 2026-02-24T18:39:48.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.023+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff31c160510 con 0x7ff31c0ccb30 2026-02-24T18:39:48.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.023+0000 7ff309ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:48.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.023+0000 7ff309ffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:48.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.123+0000 7ff323888640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:48.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.123+0000 7ff323888640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:48.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.123+0000 7ff323888640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:39:48.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.123+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff31c074850 con 0x7ff31c0ccb30 2026-02-24T18:39:48.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.125+0000 7ff309ffb640 1 -- 10.20.193.18:0/2746537841 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff30c062310 con 0x7ff31c0ccb30 2026-02-24T18:39:48.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.125+0000 7ff309ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:48.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.125+0000 7ff309ffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:48.127 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:48.127 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:48.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 10 librados: watch_flush enter 2026-02-24T18:39:48.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 10 librados: watch_flush exit 2026-02-24T18:39:48.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 20 client.5044.objecter shutdown clearing up homeless session... 2026-02-24T18:39:48.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 10 client.5044.objecter successfully canceled tick 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 msgr2=0x7ff2ec07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 0x7ff2ec07b2c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c9620 tx=0x7ff310009040 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 10 monclient: shutdown 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 msgr2=0x7ff31c1ea710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 0x7ff31c1ea710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00e870 tx=0x7ff30c00ed40 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 shutdown_connections 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff2ec078e00 0x7ff2ec07b2c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff31c156050 0x7ff31c1f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff31c155c70 0x7ff31c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 --2- 10.20.193.18:0/2746537841 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff31c0ccb30 0x7ff31c1ea710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:48.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 >> 10.20.193.18:0/2746537841 conn(0x7ff31c0cc010 msgr2=0x7ff31c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:48.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 shutdown_connections 2026-02-24T18:39:48.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 -- 10.20.193.18:0/2746537841 wait complete. 2026-02-24T18:39:48.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:48.126+0000 7ff323888640 1 librados: shutdown 2026-02-24T18:39:51.139 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.206+0000 7f1a795a9640 10 monclient: get_monmap_and_config 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.206+0000 7f1a795a9640 10 monclient: build_initial_monmap 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.206+0000 7f1a795a9640 10 monclient: monmap: 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:51.207552+0000 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:51.207552+0000 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:51.208 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 Processor -- start 2026-02-24T18:39:51.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 -- start start 2026-02-24T18:39:51.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient: init 2026-02-24T18:39:51.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:51.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient: _add_conns ranks=[0,2,5,3,1,4] 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1a7406ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1a740c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1a7415a5e0 0x7f1a74070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-c con 0x7f1a7415a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a740c7e90 con 0x7f1a7406ff20 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a740c8170 con 0x7f1a740c99a0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1a74159a40 con 0x7f1a7415a5e0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a795a9640 10 monclient(hunting): authenticate will time out at 1734.888684s 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a72ffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1a7406ff20 0x7f1a74159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:49748/0 (socket says 10.20.193.18:49748) 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.207+0000 7f1a72ffd640 1 -- 10.20.193.18:0/2442348166 learned_addr learned my addr 10.20.193.18:0/2442348166 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 1 --2- 10.20.193.18:0/2442348166 >> v2:10.20.193.110:3300/0 conn(0x7f1a7415a5e0 0x7f1a74070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): get_auth_request con 0x7f1a7415a5e0 auth_method 0 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.208+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 1 -- 10.20.193.18:0/2442348166 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481413719 0 0) 0x7f1a740c7e90 con 0x7f1a7406ff20 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:39:51.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 10 monclient(hunting): my global_id is 5079 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a71ffb640 1 -- 10.20.193.18:0/2442348166 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a60003620 con 0x7f1a7406ff20 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 10 monclient(hunting): handle_auth_done global_id 5027 payload 274 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 1 -- 10.20.193.18:0/2442348166 >> v1:10.20.193.110:6789/0 conn(0x7f1a740c99a0 legacy=0x7f1a7414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 1 -- 10.20.193.18:0/2442348166 >> v1:10.20.193.18:6789/0 conn(0x7f1a7406ff20 legacy=0x7f1a74159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 10 monclient: _finish_hunting 0 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 1 monclient: found mon.noname-c 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.209+0000 7f1a737fe640 1 -- 10.20.193.18:0/2442348166 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a740c8450 con 0x7f1a7415a5e0 2026-02-24T18:39:51.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.211+0000 7f1a737fe640 1 --2- 10.20.193.18:0/2442348166 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a74070300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009dc0 tx=0x7f1a6402f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b974de9a6b004b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 1 -- 10.20.193.18:0/2442348166 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6403c070 con 0x7f1a7415a5e0 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: dump: 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.214 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: _finish_auth 0 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 1 -- 10.20.193.18:0/2442348166 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64037440 con 0x7f1a7415a5e0 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 1 -- 10.20.193.18:0/2442348166 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64037810 con 0x7f1a7415a5e0 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 5 monclient: authenticate success, global_id 5027 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a71ffb640 10 monclient: dump: 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:51.215 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 4 set_mon_vals no callback set 2026-02-24T18:39:51.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 10 monclient: shutdown 2026-02-24T18:39:51.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:51.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 -- 10.20.193.18:0/2442348166 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 msgr2=0x7f1a74070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 --2- 10.20.193.18:0/2442348166 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a74070300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009dc0 tx=0x7f1a6402f230 comp rx=0 tx=0).stop 2026-02-24T18:39:51.216 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 -- 10.20.193.18:0/2442348166 shutdown_connections 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 --2- 10.20.193.18:0/2442348166 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a74070300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 -- 10.20.193.18:0/2442348166 >> 10.20.193.18:0/2442348166 conn(0x7f1a740cd780 msgr2=0x7f1a740cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 -- 10.20.193.18:0/2442348166 shutdown_connections 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.213+0000 7f1a795a9640 1 -- 10.20.193.18:0/2442348166 wait complete. 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient: build_initial_monmap 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient: monmap: 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.217 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 librados: starting msgr at 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 librados: starting objecter 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 Processor -- start 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 -- start start 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 librados: setting wanted keys 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 librados: calling monclient init 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient: init 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:51.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a7406ff20 0x7f1a741f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-a con 0x7f1a7406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a740c99a0 0x7f1a741f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-b con 0x7f1a740c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): picked mon.noname-c con 0x7f1a7415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74150520 con 0x7f1a7406ff20 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740c8a10 con 0x7f1a740c99a0 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:51.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741506a0 con 0x7f1a7415a5e0 2026-02-24T18:39:51.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:51.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.214+0000 7f1a795a9640 10 monclient(hunting): authenticate will time out at 1734.895940s 2026-02-24T18:39:51.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a727fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a740c99a0 0x7f1a741f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:41102/0 (socket says 10.20.193.18:41102) 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 1 -- 10.20.193.18:0/3876833404 learned_addr learned my addr 10.20.193.18:0/3876833404 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): get_auth_request con 0x7f1a7415a5e0 auth_method 0 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a727fc640 10 monclient(hunting): get_auth_request con 0x7f1a740c99a0 auth_method 0 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a727fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:51.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a72ffd640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a7406ff20 0x7f1a741f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.215+0000 7f1a737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): get_auth_request con 0x7f1a7406ff20 auth_method 0 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:51.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.216+0000 7f1a72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 10 monclient(hunting): handle_auth_done global_id 5030 payload 824 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 -- 10.20.193.18:0/3876833404 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a740c99a0 msgr2=0x7f1a741f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a740c99a0 0x7f1a741f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 -- 10.20.193.18:0/3876833404 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a7406ff20 msgr2=0x7f1a741f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a7406ff20 0x7f1a741f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 10 monclient: _finish_hunting 0 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 monclient: found mon.noname-c 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.217+0000 7f1a737fe640 1 -- 10.20.193.18:0/3876833404 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a741f4710 con 0x7f1a7415a5e0 2026-02-24T18:39:51.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.219+0000 7f1a737fe640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a6403a040 tx=0x7f1a64004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.219+0000 7f1a537fe640 10 client.?.objecter ms_handle_connect 0x7f1a7415a5e0 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.219+0000 7f1a537fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64047020 con 0x7f1a7415a5e0 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: dump: 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:51.227 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: _finish_auth 0 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: _check_auth_tickets 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64042670 con 0x7f1a7415a5e0 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6403c040 con 0x7f1a7415a5e0 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:39:51.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a537fe640 10 monclient: dump: 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a717fa640 4 set_mon_vals no callback set 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a795a9640 5 monclient: authenticate success, global_id 5030 2026-02-24T18:39:51.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a795a9640 10 monclient: _renew_subs 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a795a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.220+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a740ba430 con 0x7f1a7415a5e0 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 client.5030.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 monclient: _renew_subs 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741f4430 con 0x7f1a7415a5e0 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 1 librados: init done 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.221+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a74070300 con 0x7f1a7415a5e0 2026-02-24T18:39:51.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.232+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1a64042a40 con 0x7f1a7415a5e0 2026-02-24T18:39:51.233 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.232+0000 7f1a537fe640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 0x7f1a4007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:51.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.232+0000 7f1a72ffd640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 0x7f1a4007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:51.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.232+0000 7f1a72ffd640 10 monclient: get_auth_request con 0x7f1a40078e80 auth_method 0 2026-02-24T18:39:51.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.233+0000 7f1a72ffd640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 0x7f1a4007b340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a68007b60 tx=0x7f1a68007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:51.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.233+0000 7f1a537fe640 10 client.5030.objecter ms_handle_connect 0x7f1a40078e80 2026-02-24T18:39:51.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.234+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f1a64042d60 con 0x7f1a7415a5e0 2026-02-24T18:39:51.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.234+0000 7f1a537fe640 10 client.5030.objecter ms_dispatch2 0x7f1a74000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:51.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.234+0000 7f1a537fe640 3 client.5030.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:51.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.234+0000 7f1a537fe640 3 client.5030.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:51.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.234+0000 7f1a537fe640 20 client.5030.objecter dump_active .. 0 homeless 2026-02-24T18:39:51.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.333+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1a64051070 con 0x7f1a7415a5e0 2026-02-24T18:39:51.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.333+0000 7f1a537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:51.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.333+0000 7f1a537fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:51.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.432+0000 7f1a795a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:51.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.432+0000 7f1a795a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:51.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.432+0000 7f1a795a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:39:51.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.432+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1a7406f510 con 0x7f1a7415a5e0 2026-02-24T18:39:51.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.435+0000 7f1a537fe640 1 -- 10.20.193.18:0/3876833404 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1a64087ee0 con 0x7f1a7415a5e0 2026-02-24T18:39:51.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.435+0000 7f1a537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:51.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.435+0000 7f1a537fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:51.437 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:51.437 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:51.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 10 librados: watch_flush enter 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 10 librados: watch_flush exit 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 20 client.5030.objecter shutdown clearing up homeless session... 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 10 client.5030.objecter successfully canceled tick 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 msgr2=0x7f1a4007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 0x7f1a4007b340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a68007b60 tx=0x7f1a68007ab0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 10 monclient: shutdown 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 msgr2=0x7f1a740cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a6403a040 tx=0x7f1a64004b40 comp rx=0 tx=0).stop 2026-02-24T18:39:51.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 shutdown_connections 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a40078e80 0x7f1a4007b340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a740c99a0 0x7f1a741f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a7415a5e0 0x7f1a740cc560 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 --2- 10.20.193.18:0/3876833404 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a7406ff20 0x7f1a741f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 >> 10.20.193.18:0/3876833404 conn(0x7f1a740cd780 msgr2=0x7f1a740c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 shutdown_connections 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 -- 10.20.193.18:0/3876833404 wait complete. 2026-02-24T18:39:51.440 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:51.436+0000 7f1a795a9640 1 librados: shutdown 2026-02-24T18:39:54.447 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 10 monclient: get_monmap_and_config 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 10 monclient: build_initial_monmap 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 10 monclient: monmap: 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:54.516880+0000 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:54.516880+0000 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:54.517 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 1 Processor -- start 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 1 -- start start 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.516+0000 7f8bd86f1640 10 monclient: init 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient: _add_conns ranks=[4,0,3,1,2,5] 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f8bd0155dd0 0x7f8bd0071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-b con 0x7f8bd0155dd0 addr v2:10.20.193.32:3300/0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bd015ffc0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8bd01561b0 0x7f8bd00c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-a con 0x7f8bd01561b0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd0073960 con 0x7f8bd015ffc0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd0073710 con 0x7f8bd01561b0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd0073c40 con 0x7f8bd0155dd0 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:54.518 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd86f1640 10 monclient(hunting): authenticate will time out at 1738.198123s 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8bd01561b0 0x7f8bd00c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f8bd01561b0 0x7f8bd00c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:58182/0 (socket says 10.20.193.18:58182) 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 1 -- 10.20.193.18:0/3416708326 learned_addr learned my addr 10.20.193.18:0/3416708326 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): get_auth_request con 0x7f8bd01561b0 auth_method 0 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6466640 1 --2- 10.20.193.18:0/3416708326 >> v2:10.20.193.32:3300/0 conn(0x7f8bd0155dd0 0x7f8bd0071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6466640 10 monclient(hunting): get_auth_request con 0x7f8bd0155dd0 auth_method 0 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6466640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.517+0000 7f8bd6466640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:54.519 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_done global_id 5085 payload 274 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 1 -- 10.20.193.18:0/3416708326 >> v2:10.20.193.32:3300/0 conn(0x7f8bd0155dd0 msgr2=0x7f8bd0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 1 --2- 10.20.193.18:0/3416708326 >> v2:10.20.193.32:3300/0 conn(0x7f8bd0155dd0 0x7f8bd0071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 1 -- 10.20.193.18:0/3416708326 >> v1:10.20.193.18:6789/0 conn(0x7f8bd015ffc0 legacy=0x7f8bd0076250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 10 monclient: _finish_hunting 0 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 1 monclient: found mon.noname-a 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.520 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.518+0000 7f8bd6c67640 1 -- 10.20.193.18:0/3416708326 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0073f20 con 0x7f8bd01561b0 2026-02-24T18:39:54.521 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.519+0000 7f8bd6c67640 1 --2- 10.20.193.18:0/3416708326 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd01561b0 0x7f8bd00c9020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072850 tx=0x7f8bcc02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbd44a4c8d8c36d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 1 -- 10.20.193.18:0/3416708326 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02fc50 con 0x7f8bd01561b0 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 10 monclient: dump: 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 10 monclient: _finish_auth 0 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 10 monclient: _check_auth_tickets 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.521+0000 7f8bd5464640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 5 monclient: authenticate success, global_id 5085 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 1 -- 10.20.193.18:0/3416708326 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc02fdf0 con 0x7f8bd01561b0 2026-02-24T18:39:54.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 1 -- 10.20.193.18:0/3416708326 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc037610 con 0x7f8bd01561b0 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd5464640 10 monclient: dump: 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:54.524 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:54.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 4 set_mon_vals no callback set 2026-02-24T18:39:54.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 10 monclient: shutdown 2026-02-24T18:39:54.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:54.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 -- 10.20.193.18:0/3416708326 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd01561b0 msgr2=0x7f8bd00c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:54.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/3416708326 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd01561b0 0x7f8bd00c9020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072850 tx=0x7f8bcc02f230 comp rx=0 tx=0).stop 2026-02-24T18:39:54.529 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 -- 10.20.193.18:0/3416708326 shutdown_connections 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/3416708326 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd01561b0 0x7f8bd00c9020 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/3416708326 >> v2:10.20.193.32:3300/0 conn(0x7f8bd0155dd0 0x7f8bd0071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 -- 10.20.193.18:0/3416708326 >> 10.20.193.18:0/3416708326 conn(0x7f8bd00ca710 msgr2=0x7f8bd00cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 -- 10.20.193.18:0/3416708326 shutdown_connections 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 1 -- 10.20.193.18:0/3416708326 wait complete. 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.522+0000 7f8bd86f1640 10 monclient: build_initial_monmap 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient: monmap: 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:54.530 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 librados: starting msgr at 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 librados: starting objecter 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 Processor -- start 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 -- start start 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 librados: setting wanted keys 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 librados: calling monclient init 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient: init 2026-02-24T18:39:54.531 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8bd0155dd0 0x7f8bd01f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-c con 0x7f8bd0155dd0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8bd01561b0 0x7f8bd00bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-b con 0x7f8bd01561b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): picked mon.noname-a con 0x7f8bd015ffc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00c9560 con 0x7f8bd015ffc0 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00c8b00 con 0x7f8bd01561b0 2026-02-24T18:39:54.532 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00c96e0 con 0x7f8bd0155dd0 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd86f1640 10 monclient(hunting): authenticate will time out at 1738.204731s 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.523+0000 7f8bd6c67640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:58198/0 (socket says 10.20.193.18:58198) 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 1 -- 10.20.193.18:0/2629476406 learned_addr learned my addr 10.20.193.18:0/2629476406 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): get_auth_request con 0x7f8bd015ffc0 auth_method 0 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8bd01561b0 0x7f8bd00bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8bd0155dd0 0x7f8bd01f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.533 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): get_auth_request con 0x7f8bd01561b0 auth_method 0 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): get_auth_request con 0x7f8bd0155dd0 auth_method 0 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:54.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd5c65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.524+0000 7f8bd6466640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 10 monclient(hunting): handle_auth_done global_id 5088 payload 824 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 -- 10.20.193.18:0/2629476406 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8bd0155dd0 msgr2=0x7f8bd01f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8bd0155dd0 0x7f8bd01f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 -- 10.20.193.18:0/2629476406 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8bd01561b0 msgr2=0x7f8bd00bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8bd01561b0 0x7f8bd00bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 10 monclient: _finish_hunting 0 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 monclient: found mon.noname-a 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:54.535 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.525+0000 7f8bd6c67640 1 -- 10.20.193.18:0/2629476406 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd00c0640 con 0x7f8bd015ffc0 2026-02-24T18:39:54.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.526+0000 7f8bd6c67640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc039040 tx=0x7f8bcc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:54.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.526+0000 7f8bb6ffd640 10 client.?.objecter ms_handle_connect 0x7f8bd015ffc0 2026-02-24T18:39:54.536 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.526+0000 7f8bb6ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:54.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc046070 con 0x7f8bd015ffc0 2026-02-24T18:39:54.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: dump: 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:54.538 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc042660 con 0x7f8bd015ffc0 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 5 monclient: authenticate success, global_id 5088 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc042a50 con 0x7f8bd015ffc0 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd4c63640 4 set_mon_vals no callback set 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bb6ffd640 10 monclient: dump: 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:54.539 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _renew_subs 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd00c0920 con 0x7f8bd015ffc0 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 client.5088.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:54.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _renew_subs 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd00c0ee0 con 0x7f8bd015ffc0 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 1 librados: init done 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.541 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.529+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd0071360 con 0x7f8bd015ffc0 2026-02-24T18:39:54.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.637+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8bcc053060 con 0x7f8bd015ffc0 2026-02-24T18:39:54.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.638+0000 7f8bb6ffd640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 0x7f8ba007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:54.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.638+0000 7f8bd6466640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 0x7f8ba007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:54.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.638+0000 7f8bd6466640 10 monclient: get_auth_request con 0x7f8ba0078e50 auth_method 0 2026-02-24T18:39:54.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.638+0000 7f8bd6466640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 0x7f8ba007b310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00bf2d0 tx=0x7f8bc0009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:54.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.638+0000 7f8bb6ffd640 10 client.5088.objecter ms_handle_connect 0x7f8ba0078e50 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f8bcc083680 con 0x7f8bd015ffc0 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 10 client.5088.objecter ms_dispatch2 0x7f8bd0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 3 client.5088.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 3 client.5088.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 20 client.5088.objecter dump_active .. 0 homeless 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8bcc087640 con 0x7f8bd015ffc0 2026-02-24T18:39:54.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:54.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.646+0000 7f8bb6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:54.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.749+0000 7f8bd86f1640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:54.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.749+0000 7f8bd86f1640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:54.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.749+0000 7f8bd86f1640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:54.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.749+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8bd0074bd0 con 0x7f8bd015ffc0 2026-02-24T18:39:54.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.751+0000 7f8bb6ffd640 1 -- 10.20.193.18:0/2629476406 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8bcc08b940 con 0x7f8bd015ffc0 2026-02-24T18:39:54.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.751+0000 7f8bb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:54.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.751+0000 7f8bb6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:54.753 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:54.753 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 10 librados: watch_flush enter 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 10 librados: watch_flush exit 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 20 client.5088.objecter shutdown clearing up homeless session... 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 10 client.5088.objecter successfully canceled tick 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 msgr2=0x7f8ba007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 0x7f8ba007b310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00bf2d0 tx=0x7f8bc0009290 comp rx=0 tx=0).stop 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 10 monclient: shutdown 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:54.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 msgr2=0x7f8bd014e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc039040 tx=0x7f8bcc004b40 comp rx=0 tx=0).stop 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 shutdown_connections 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8ba0078e50 0x7f8ba007b310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f8bd01561b0 0x7f8bd00bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f8bd015ffc0 0x7f8bd014e700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 --2- 10.20.193.18:0/2629476406 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f8bd0155dd0 0x7f8bd01f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 >> 10.20.193.18:0/2629476406 conn(0x7f8bd00ca710 msgr2=0x7f8bd0150070 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.752+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 shutdown_connections 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.753+0000 7f8bd86f1640 1 -- 10.20.193.18:0/2629476406 wait complete. 2026-02-24T18:39:54.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:54.753+0000 7f8bd86f1640 1 librados: shutdown 2026-02-24T18:39:57.765 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 10 monclient: get_monmap_and_config 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 10 monclient: build_initial_monmap 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 10 monclient: monmap: 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:39:57.835961+0000 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:39:57.835961+0000 2026-02-24T18:39:57.837 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 1 Processor -- start 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 1 -- start start 2026-02-24T18:39:57.838 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.835+0000 7f3129197640 10 monclient: init 2026-02-24T18:39:57.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:57.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient: _add_conns ranks=[2,0,3,4,5,1] 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f312406ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f31240c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f312415a5e0 0x7f3124070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): picked mon.noname-a con 0x7f312415a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31240c7e90 con 0x7f31240c99a0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31240c8170 con 0x7f312406ff20 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3124159a40 con 0x7f312415a5e0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3129197640 10 monclient(hunting): authenticate will time out at 1741.517095s 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f312415a5e0 0x7f3124070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:57.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f312415a5e0 0x7f3124070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:58214/0 (socket says 10.20.193.18:58214) 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 1 -- 10.20.193.18:0/1001215874 learned_addr learned my addr 10.20.193.18:0/1001215874 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): get_auth_request con 0x7f312415a5e0 auth_method 0 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.836+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 10 monclient(hunting): handle_auth_done global_id 5091 payload 274 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 1 -- 10.20.193.18:0/1001215874 >> v1:10.20.193.110:6789/0 conn(0x7f312406ff20 legacy=0x7f3124159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 1 -- 10.20.193.18:0/1001215874 >> v1:10.20.193.18:6789/0 conn(0x7f31240c99a0 legacy=0x7f312414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 10 monclient: _finish_hunting 0 2026-02-24T18:39:57.842 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 1 monclient: found mon.noname-a 2026-02-24T18:39:57.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:57.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:57.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.837+0000 7f3123577640 1 -- 10.20.193.18:0/1001215874 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31240c8450 con 0x7f312415a5e0 2026-02-24T18:39:57.843 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.838+0000 7f3123577640 1 --2- 10.20.193.18:0/1001215874 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f3124070300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f311402f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140693caa787ae3a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 1 -- 10.20.193.18:0/1001215874 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311402fd40 con 0x7f312415a5e0 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: dump: 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:57.848 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: _finish_auth 0 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: _check_auth_tickets 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 1 -- 10.20.193.18:0/1001215874 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31140373f0 con 0x7f312415a5e0 2026-02-24T18:39:57.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3129197640 5 monclient: authenticate success, global_id 5091 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 1 -- 10.20.193.18:0/1001215874 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140377c0 con 0x7f312415a5e0 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3121d74640 10 monclient: dump: 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:57.850 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3129197640 10 monclient: get_monmap_and_config success 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.840+0000 7f3129197640 4 set_mon_vals no callback set 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 10 monclient: shutdown 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- 10.20.193.18:0/1001215874 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 msgr2=0x7f3124070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 --2- 10.20.193.18:0/1001215874 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f3124070300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f311402f2d0 comp rx=0 tx=0).stop 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- 10.20.193.18:0/1001215874 shutdown_connections 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 --2- 10.20.193.18:0/1001215874 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f3124070300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- 10.20.193.18:0/1001215874 >> 10.20.193.18:0/1001215874 conn(0x7f31240cd780 msgr2=0x7f31240cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- 10.20.193.18:0/1001215874 shutdown_connections 2026-02-24T18:39:57.851 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- 10.20.193.18:0/1001215874 wait complete. 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 10 monclient: build_initial_monmap 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 10 monclient: monmap: 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:39:57.852 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 librados: starting msgr at 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 librados: starting objecter 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 Processor -- start 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 -- start start 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 librados: setting wanted keys 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 1 librados: calling monclient init 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.841+0000 7f3129197640 10 monclient: init 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient: _reopen_session rank -1 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f312406ff20 0x7f3124156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): picked mon.noname-b con 0x7f312406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f31240c99a0 0x7f3124156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:57.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): picked mon.noname-c con 0x7f31240c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): picked mon.noname-a con 0x7f312415a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3124150520 con 0x7f312415a5e0 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f31240c8a10 con 0x7f312406ff20 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): start opening mon connection 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f31241506a0 con 0x7f31240c99a0 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): _renew_subs 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3129197640 10 monclient(hunting): authenticate will time out at 1741.523374s 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:57.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:58224/0 (socket says 10.20.193.18:58224) 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 1 -- 10.20.193.18:0/1560086661 learned_addr learned my addr 10.20.193.18:0/1560086661 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 10 monclient(hunting): get_auth_request con 0x7f312415a5e0 auth_method 0 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3123577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3122d76640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f312406ff20 0x7f3124156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.842+0000 7f3122575640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f31240c99a0 0x7f3124156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): get_auth_request con 0x7f312406ff20 auth_method 0 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:57.855 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3123577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): get_auth_request con 0x7f31240c99a0 auth_method 0 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.843+0000 7f3122d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:39:57.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 10 monclient(hunting): handle_auth_done global_id 5094 payload 824 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 -- 10.20.193.18:0/1560086661 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f31240c99a0 msgr2=0x7f3124156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f31240c99a0 0x7f3124156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 -- 10.20.193.18:0/1560086661 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f312406ff20 msgr2=0x7f3124156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f312406ff20 0x7f3124156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 10 monclient: _finish_hunting 0 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 monclient: found mon.noname-a 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.844+0000 7f3123577640 1 -- 10.20.193.18:0/1560086661 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31241f7c10 con 0x7f312415a5e0 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.845+0000 7f3123577640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f3114004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.845+0000 7f31037fe640 10 client.?.objecter ms_handle_connect 0x7f312415a5e0 2026-02-24T18:39:57.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.845+0000 7f31037fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:39:57.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114046020 con 0x7f312415a5e0 2026-02-24T18:39:57.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: dump: 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:57.860 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: _finish_auth 0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: _check_auth_tickets 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3114037de0 con 0x7f312415a5e0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 5 monclient: authenticate success, global_id 5094 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 10 monclient: _renew_subs 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31241e98e0 con 0x7f312415a5e0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114041690 con 0x7f312415a5e0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:39:57.861 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3121573640 4 set_mon_vals no callback set 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f31037fe640 10 monclient: dump: 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:39:57.862 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 10 client.5094.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 10 monclient: _renew_subs 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241f1cd0 con 0x7f312415a5e0 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.847+0000 7f3129197640 1 librados: init done 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.848+0000 7f3129197640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.848+0000 7f3129197640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.848+0000 7f3129197640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:57.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.848+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3124074a80 con 0x7f312415a5e0 2026-02-24T18:39:57.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.859+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3114041830 con 0x7f312415a5e0 2026-02-24T18:39:57.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.860+0000 7f31037fe640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 0x7f30f007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:39:57.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.860+0000 7f3122d76640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 0x7f30f007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.860+0000 7f3122d76640 10 monclient: get_auth_request con 0x7f30f0078e50 auth_method 0 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.861+0000 7f3122d76640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 0x7f30f007b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31180051f0 tx=0x7f3118007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.861+0000 7f31037fe640 10 client.5094.objecter ms_handle_connect 0x7f30f0078e50 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.862+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f31140839f0 con 0x7f312415a5e0 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.862+0000 7f31037fe640 10 client.5094.objecter ms_dispatch2 0x7f3124000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.862+0000 7f31037fe640 3 client.5094.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.862+0000 7f31037fe640 3 client.5094.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:39:57.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.862+0000 7f31037fe640 20 client.5094.objecter dump_active .. 0 homeless 2026-02-24T18:39:57.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.976+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3124074a80 con 0x7f312415a5e0 2026-02-24T18:39:57.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.976+0000 7f31037fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:39:57.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:57.976+0000 7f31037fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:39:58.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.076+0000 7f3129197640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:39:58.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.076+0000 7f3129197640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:58.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.076+0000 7f3129197640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:39:58.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.076+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3124070300 con 0x7f312415a5e0 2026-02-24T18:39:58.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.079+0000 7f31037fe640 1 -- 10.20.193.18:0/1560086661 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3114050030 con 0x7f312415a5e0 2026-02-24T18:39:58.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.079+0000 7f31037fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:39:58.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.079+0000 7f31037fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:39:58.081 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:39:58.081 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 10 librados: watch_flush enter 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 10 librados: watch_flush exit 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 20 client.5094.objecter shutdown clearing up homeless session... 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 10 client.5094.objecter successfully canceled tick 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 msgr2=0x7f30f007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:58.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 0x7f30f007b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31180051f0 tx=0x7f3118007f30 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 10 monclient: shutdown 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 msgr2=0x7f31241f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3114009e80 tx=0x7f3114004b40 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 shutdown_connections 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f30f0078e50 0x7f30f007b310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f312415a5e0 0x7f31241f6da0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f31240c99a0 0x7f3124156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 --2- 10.20.193.18:0/1560086661 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f312406ff20 0x7f3124156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 >> 10.20.193.18:0/1560086661 conn(0x7f31240cd780 msgr2=0x7f31240c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 shutdown_connections 2026-02-24T18:39:58.084 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 -- 10.20.193.18:0/1560086661 wait complete. 2026-02-24T18:39:58.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:39:58.080+0000 7f3129197640 1 librados: shutdown 2026-02-24T18:39:59.692 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:37.638 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.694 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:37.640 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.701 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:37.647 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.718 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:37.664 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.720 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:37.666 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.722 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:37.630 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.723 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:37.631 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.728 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:37.674 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:39:59.731 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:37.639 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:01.092 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: get_monmap_and_config 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: build_initial_monmap 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: monmap: 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:01.163903+0000 2026-02-24T18:40:01.165 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:01.163903+0000 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 1 Processor -- start 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 1 -- start start 2026-02-24T18:40:01.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: init 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.163+0000 7fbcd9b53640 10 monclient: _add_conns ranks=[0,2,5,3,4,1] 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbcd406ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fbcd40c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fbcd415a5e0 0x7fbcd4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-c con 0x7fbcd415a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd40c7e90 con 0x7fbcd406ff20 2026-02-24T18:40:01.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd40c8170 con 0x7fbcd40c99a0 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd4159a40 con 0x7fbcd415a5e0 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd9b53640 10 monclient(hunting): authenticate will time out at 1744.845054s 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd37fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fbcd406ff20 0x7fbcd4159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:54556/0 (socket says 10.20.193.18:54556) 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd37fe640 1 -- 10.20.193.18:0/2034321187 learned_addr learned my addr 10.20.193.18:0/2034321187 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd3fff640 1 --2- 10.20.193.18:0/2034321187 >> v2:10.20.193.110:3300/0 conn(0x7fbcd415a5e0 0x7fbcd4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd3fff640 10 monclient(hunting): get_auth_request con 0x7fbcd415a5e0 auth_method 0 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.164+0000 7fbcd3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:01.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 1 -- 10.20.193.18:0/2034321187 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302606137 0 0) 0x7fbcd40c7e90 con 0x7fbcd406ff20 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 10 monclient(hunting): _negotiate 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 10 monclient(hunting): my global_id is 5097 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.165+0000 7fbcd27fc640 1 -- 10.20.193.18:0/2034321187 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb8003620 con 0x7fbcd406ff20 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_done global_id 5042 payload 274 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2034321187 >> v1:10.20.193.110:6789/0 conn(0x7fbcd40c99a0 legacy=0x7fbcd414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2034321187 >> v1:10.20.193.18:6789/0 conn(0x7fbcd406ff20 legacy=0x7fbcd4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:01.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 1 monclient: found mon.noname-c 2026-02-24T18:40:01.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:01.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.166+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2034321187 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd40c8450 con 0x7fbcd415a5e0 2026-02-24T18:40:01.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.167+0000 7fbcd3fff640 1 --2- 10.20.193.18:0/2034321187 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd4070300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009dc0 tx=0x7fbcc802f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1462effc8569750 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 1 -- 10.20.193.18:0/2034321187 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803c070 con 0x7fbcd415a5e0 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: dump: 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.177 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 5 monclient: authenticate success, global_id 5042 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 1 -- 10.20.193.18:0/2034321187 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc8037440 con 0x7fbcd415a5e0 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:01.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 1 -- 10.20.193.18:0/2034321187 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8037810 con 0x7fbcd415a5e0 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 4 set_mon_vals no callback set 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd27fc640 10 monclient: dump: 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:01.179 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 10 monclient: shutdown 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2034321187 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 msgr2=0x7fbcd4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2034321187 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd4070300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009dc0 tx=0x7fbcc802f230 comp rx=0 tx=0).stop 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2034321187 shutdown_connections 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2034321187 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd4070300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2034321187 >> 10.20.193.18:0/2034321187 conn(0x7fbcd40cd780 msgr2=0x7fbcd40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2034321187 shutdown_connections 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.170+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2034321187 wait complete. 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 10 monclient: build_initial_monmap 2026-02-24T18:40:01.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 10 monclient: monmap: 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 librados: starting msgr at 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 librados: starting objecter 2026-02-24T18:40:01.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 Processor -- start 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 -- start start 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 librados: setting wanted keys 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 1 librados: calling monclient init 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.171+0000 7fbcd9b53640 10 monclient: init 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 0x7fbcd41f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-a con 0x7fbcd406ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbcd40c99a0 0x7fbcd40ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-b con 0x7fbcd40c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:01.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): picked mon.noname-c con 0x7fbcd415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4150520 con 0x7fbcd406ff20 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd40c8a10 con 0x7fbcd40c99a0 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41506a0 con 0x7fbcd415a5e0 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd9b53640 10 monclient(hunting): authenticate will time out at 1744.853238s 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 0x7fbcd41f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44588/0 (socket says 10.20.193.18:44588) 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 0x7fbcd41f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44398/0 (socket says 10.20.193.18:44398) 2026-02-24T18:40:01.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 1 -- 10.20.193.18:0/2948157752 learned_addr learned my addr 10.20.193.18:0/2948157752 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): get_auth_request con 0x7fbcd406ff20 auth_method 0 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): get_auth_request con 0x7fbcd415a5e0 auth_method 0 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:01.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd2ffd640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbcd40c99a0 0x7fbcd40ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.172+0000 7fbcd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): get_auth_request con 0x7fbcd40c99a0 auth_method 0 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.173+0000 7fbcd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 10 monclient(hunting): handle_auth_done global_id 5045 payload 824 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2948157752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbcd40c99a0 msgr2=0x7fbcd40ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbcd40c99a0 0x7fbcd40ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2948157752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 msgr2=0x7fbcd41f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 0x7fbcd41f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 monclient: found mon.noname-c 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.174+0000 7fbcd3fff640 1 -- 10.20.193.18:0/2948157752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd40ccb80 con 0x7fbcd415a5e0 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.175+0000 7fbcd3fff640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036000 tx=0x7fbcc8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.175+0000 7fbcaffff640 10 client.?.objecter ms_handle_connect 0x7fbcd415a5e0 2026-02-24T18:40:01.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.175+0000 7fbcaffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:01.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8047030 con 0x7fbcd415a5e0 2026-02-24T18:40:01.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:01.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 10 monclient: dump: 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 10 monclient: _finish_auth 0 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcaffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 5 monclient: authenticate success, global_id 5045 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _renew_subs 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd40cce60 con 0x7fbcd415a5e0 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 client.5045.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _renew_subs 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41f7050 con 0x7fbcd415a5e0 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 1 librados: init done 2026-02-24T18:40:01.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.176+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd4070300 con 0x7fbcd415a5e0 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc8037440 con 0x7fbcd415a5e0 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803c040 con 0x7fbcd415a5e0 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcaffff640 10 monclient: dump: 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.191 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:01.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.177+0000 7fbcd1ffb640 4 set_mon_vals no callback set 2026-02-24T18:40:01.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.281+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbcc8042670 con 0x7fbcd415a5e0 2026-02-24T18:40:01.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.282+0000 7fbcaffff640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 0x7fbca407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:01.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.282+0000 7fbcd37fe640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 0x7fbca407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:01.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.282+0000 7fbcd37fe640 10 monclient: get_auth_request con 0x7fbca4078e00 auth_method 0 2026-02-24T18:40:01.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.282+0000 7fbcd37fe640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 0x7fbca407b2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0005980 tx=0x7fbcc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:01.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.282+0000 7fbcaffff640 10 client.5045.objecter ms_handle_connect 0x7fbca4078e00 2026-02-24T18:40:01.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.284+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fbcc80bd260 con 0x7fbcd415a5e0 2026-02-24T18:40:01.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.284+0000 7fbcaffff640 10 client.5045.objecter ms_dispatch2 0x7fbcd4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:01.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.284+0000 7fbcaffff640 3 client.5045.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:01.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.284+0000 7fbcaffff640 3 client.5045.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:01.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.284+0000 7fbcaffff640 20 client.5045.objecter dump_active .. 0 homeless 2026-02-24T18:40:01.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.290+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbcc8087a30 con 0x7fbcd415a5e0 2026-02-24T18:40:01.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.290+0000 7fbcaffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:01.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.290+0000 7fbcaffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:01.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.388+0000 7fbcd9b53640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:01.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.388+0000 7fbcd9b53640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:01.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.388+0000 7fbcd9b53640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:01.389 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.388+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbcd40cdd00 con 0x7fbcd415a5e0 2026-02-24T18:40:01.391 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcaffff640 1 -- 10.20.193.18:0/2948157752 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbcc808bd30 con 0x7fbcd415a5e0 2026-02-24T18:40:01.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcaffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:01.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcaffff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:01.392 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:01.392 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 10 librados: watch_flush enter 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 10 librados: watch_flush exit 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 20 client.5045.objecter shutdown clearing up homeless session... 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 10 client.5045.objecter successfully canceled tick 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 msgr2=0x7fbca407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 0x7fbca407b2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0005980 tx=0x7fbcc0009040 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 10 monclient: shutdown 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 msgr2=0x7fbcd40bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036000 tx=0x7fbcc8004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 shutdown_connections 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fbca4078e00 0x7fbca407b2c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fbcd40c99a0 0x7fbcd40ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fbcd415a5e0 0x7fbcd40bcb70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 --2- 10.20.193.18:0/2948157752 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fbcd406ff20 0x7fbcd41f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 >> 10.20.193.18:0/2948157752 conn(0x7fbcd40cd780 msgr2=0x7fbcd40728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 shutdown_connections 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 -- 10.20.193.18:0/2948157752 wait complete. 2026-02-24T18:40:01.393 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:01.391+0000 7fbcd9b53640 1 librados: shutdown 2026-02-24T18:40:04.402 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:04.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.472+0000 7f4de3238640 10 monclient: get_monmap_and_config 2026-02-24T18:40:04.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.472+0000 7f4de3238640 10 monclient: build_initial_monmap 2026-02-24T18:40:04.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.472+0000 7f4de3238640 10 monclient: monmap: 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:04.473636+0000 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:04.473636+0000 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:04.475 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:04.476 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 1 Processor -- start 2026-02-24T18:40:04.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 1 -- start start 2026-02-24T18:40:04.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient: init 2026-02-24T18:40:04.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:04.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient: _add_conns ranks=[0,2,1,4,5,3] 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4ddc06ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4ddc0c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4ddc15a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc0c7e90 con 0x7f4ddc06ff20 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc0c8170 con 0x7f4ddc15a5e0 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc0c8450 con 0x7f4ddc0c99a0 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.473+0000 7f4de3238640 10 monclient(hunting): authenticate will time out at 1748.154802s 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.474+0000 7f4de0fad640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f4ddc06ff20 0x7f4ddc159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:54558/0 (socket says 10.20.193.18:54558) 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.474+0000 7f4de0fad640 1 -- 10.20.193.18:0/830043191 learned_addr learned my addr 10.20.193.18:0/830043191 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:04.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274302041 0 0) 0x7f4ddc0c7e90 con 0x7f4ddc06ff20 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _negotiate 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): my global_id is 5103 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dc8003620 con 0x7f4ddc06ff20 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666989377 0 0) 0x7f4ddc0c8450 con 0x7f4ddc0c99a0 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _negotiate 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): my global_id is 5048 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ddc0c7e90 con 0x7f4ddc0c99a0 2026-02-24T18:40:04.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511753380 0 0) 0x7f4ddc0c8170 con 0x7f4ddc15a5e0 2026-02-24T18:40:04.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _negotiate 2026-02-24T18:40:04.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 10 monclient(hunting): my global_id is 5062 2026-02-24T18:40:04.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.475+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ddc0c8450 con 0x7f4ddc15a5e0 2026-02-24T18:40:04.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.0 v1:10.20.193.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 587745904 0 0) 0x7f4dc8003620 con 0x7f4ddc06ff20 2026-02-24T18:40:04.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 >> v1:10.20.193.110:6789/0 conn(0x7f4ddc0c99a0 legacy=0x7f4ddc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 >> v1:10.20.193.32:6789/0 conn(0x7f4ddc15a5e0 legacy=0x7f4ddc0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 monclient: found mon.noname-a-legacy 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.18:6789/0 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> v1:10.20.193.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc0c8730 con 0x7f4ddc06ff20 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.0 v1:10.20.193.18:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f4dcc002f00 con 0x7f4ddc06ff20 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: dump: 2026-02-24T18:40:04.490 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] but i'm connected to v1:10.20.193.18:6789/0, reconnecting 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:04.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 >> v1:10.20.193.18:6789/0 conn(0x7f4ddc06ff20 legacy=0x7f4ddc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 0x7f4dc8004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): picked mon.c con 0x7f4dc8004010 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004e40 0x7f4dc80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): picked mon.a con 0x7f4dc8004e40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4dc8009ab0 0x7f4dc800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): picked mon.b con 0x7f4dc8009ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800c570 con 0x7f4dc8004e40 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800c720 con 0x7f4dc8009ab0 2026-02-24T18:40:04.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800c920 con 0x7f4dc8004010 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.477+0000 7f4dd37fe640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 0x7f4dc8004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): get_auth_request con 0x7f4dc8004010 auth_method 0 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de17ae640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4dc8009ab0 0x7f4dc800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4dd3fff640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004e40 0x7f4dc80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de17ae640 10 monclient(hunting): get_auth_request con 0x7f4dc8009ab0 auth_method 0 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de17ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de17ae640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4de17ae640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4dd3fff640 10 monclient(hunting): get_auth_request con 0x7f4dc8004e40 auth_method 0 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4dd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4dd3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.478+0000 7f4dd3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:04.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 10 monclient(hunting): handle_auth_done global_id 5103 payload 293 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 -- 10.20.193.18:0/830043191 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4dc8009ab0 msgr2=0x7f4dc800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4dc8009ab0 0x7f4dc800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 -- 10.20.193.18:0/830043191 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004e40 msgr2=0x7f4dc80052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004e40 0x7f4dc80052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 10 monclient: _finish_hunting 0 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 monclient: found mon.c 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 1 -- 10.20.193.18:0/830043191 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc8003620 con 0x7f4dc8004010 2026-02-24T18:40:04.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 10 monclient: _finish_auth 0 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 10 monclient: _check_auth_tickets 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de0fad640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de17ae640 1 -- 10.20.193.18:0/830043191 reap_dead start 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de3238640 5 monclient: authenticate success, global_id 5103 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.479+0000 7f4de3238640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.480+0000 7f4de0fad640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 0x7f4dc8004900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00c240 tx=0x7f4dcc02f530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159fb3def935b50 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc007d20 con 0x7f4dc8004010 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: dump: 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:04.496 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc005bf0 con 0x7f4dc8004010 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:04.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 1 -- 10.20.193.18:0/830043191 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc007070 con 0x7f4dc8004010 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 4 set_mon_vals no callback set 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4dd37fe640 10 monclient: dump: 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:04.498 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 10 monclient: shutdown 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 -- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 msgr2=0x7f4dc8004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 0x7f4dc8004900 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00c240 tx=0x7f4dcc02f530 comp rx=0 tx=0).stop 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 -- 10.20.193.18:0/830043191 shutdown_connections 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 --2- 10.20.193.18:0/830043191 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4dc8004010 0x7f4dc8004900 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 -- 10.20.193.18:0/830043191 >> 10.20.193.18:0/830043191 conn(0x7f4ddc0cd780 msgr2=0x7f4ddc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 -- 10.20.193.18:0/830043191 shutdown_connections 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.482+0000 7f4de3238640 1 -- 10.20.193.18:0/830043191 wait complete. 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 10 monclient: build_initial_monmap 2026-02-24T18:40:04.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 10 monclient: monmap: 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 librados: starting msgr at 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 librados: starting objecter 2026-02-24T18:40:04.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 Processor -- start 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 -- start start 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 librados: setting wanted keys 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 1 librados: calling monclient init 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 10 monclient: init 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.483+0000 7f4de3238640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 0x7f4ddc146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-a con 0x7f4dc8004010 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 0x7f4ddc14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-c con 0x7f4ddc146cb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ddc1471a0 0x7f4ddc14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): picked mon.noname-b con 0x7f4ddc1471a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:04.501 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc070300 con 0x7f4dc8004010 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc159a40 con 0x7f4ddc1471a0 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc070480 con 0x7f4ddc146cb0 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de3238640 10 monclient(hunting): authenticate will time out at 1748.165035s 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 0x7f4ddc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 0x7f4ddc14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 0x7f4ddc146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44418/0 (socket says 10.20.193.18:44418) 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 1 -- 10.20.193.18:0/2181291876 learned_addr learned my addr 10.20.193.18:0/2181291876 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): get_auth_request con 0x7f4dc8004010 auth_method 0 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ddc1471a0 0x7f4ddc14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): get_auth_request con 0x7f4ddc146cb0 auth_method 0 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de0fad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): get_auth_request con 0x7f4ddc1471a0 auth_method 0 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.484+0000 7f4de17ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 10 monclient(hunting): handle_auth_done global_id 5051 payload 824 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 -- 10.20.193.18:0/2181291876 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ddc1471a0 msgr2=0x7f4ddc14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ddc1471a0 0x7f4ddc14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 -- 10.20.193.18:0/2181291876 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 msgr2=0x7f4ddc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 0x7f4ddc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 monclient: found mon.noname-c 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.485+0000 7f4dd3fff640 1 -- 10.20.193.18:0/2181291876 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc1ef950 con 0x7f4ddc146cb0 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.486+0000 7f4dd3fff640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 0x7f4ddc14b5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400eaa0 tx=0x7f4dc400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.486+0000 7f4dd17fa640 10 client.?.objecter ms_handle_connect 0x7f4ddc146cb0 2026-02-24T18:40:04.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.486+0000 7f4dd17fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4006ba0 con 0x7f4ddc146cb0 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: dump: 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:04.508 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: _finish_auth 0 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: _check_auth_tickets 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:04.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc4006d40 con 0x7f4ddc146cb0 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc400c780 con 0x7f4ddc146cb0 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd17fa640 10 monclient: dump: 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:04.510 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4dd2ffd640 4 set_mon_vals no callback set 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 5 monclient: authenticate success, global_id 5051 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 10 monclient: _renew_subs 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc1efc30 con 0x7f4ddc146cb0 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 10 client.5051.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 10 monclient: _renew_subs 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1f01f0 con 0x7f4ddc146cb0 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.488+0000 7f4de3238640 1 librados: init done 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.489+0000 7f4de3238640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.489+0000 7f4de3238640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:04.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.489+0000 7f4de3238640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.489+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc0c6510 con 0x7f4ddc146cb0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.500+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4dc4024070 con 0x7f4ddc146cb0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.501+0000 7f4dd17fa640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 0x7f4dac07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.501+0000 7f4de0fad640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 0x7f4dac07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.501+0000 7f4de0fad640 10 monclient: get_auth_request con 0x7f4dac078e50 auth_method 0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.501+0000 7f4de0fad640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 0x7f4dac07b310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc02f570 tx=0x7f4dcc009070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:04.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.501+0000 7f4dd17fa640 10 client.5051.objecter ms_handle_connect 0x7f4dac078e50 2026-02-24T18:40:04.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.595+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f4dc405d9e0 con 0x7f4ddc146cb0 2026-02-24T18:40:04.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.595+0000 7f4dd17fa640 10 client.5051.objecter ms_dispatch2 0x7f4ddc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:04.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.595+0000 7f4dd17fa640 3 client.5051.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:04.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.595+0000 7f4dd17fa640 3 client.5051.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:04.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.595+0000 7f4dd17fa640 20 client.5051.objecter dump_active .. 0 homeless 2026-02-24T18:40:04.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.600+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4ddc0c6510 con 0x7f4ddc146cb0 2026-02-24T18:40:04.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.600+0000 7f4dd17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:04.602 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.600+0000 7f4dd17fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:04.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.698+0000 7f4de3238640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:04.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.698+0000 7f4de3238640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:04.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.698+0000 7f4de3238640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:04.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.698+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4ddc159ed0 con 0x7f4ddc146cb0 2026-02-24T18:40:04.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.701+0000 7f4dd17fa640 1 -- 10.20.193.18:0/2181291876 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4dc4021020 con 0x7f4ddc146cb0 2026-02-24T18:40:04.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.701+0000 7f4dd17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:04.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.701+0000 7f4dd17fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:04.704 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:04.704 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:04.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 10 librados: watch_flush enter 2026-02-24T18:40:04.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 10 librados: watch_flush exit 2026-02-24T18:40:04.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 20 client.5051.objecter shutdown clearing up homeless session... 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 10 client.5051.objecter successfully canceled tick 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 msgr2=0x7f4dac07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 0x7f4dac07b310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc02f570 tx=0x7f4dcc009070 comp rx=0 tx=0).stop 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 10 monclient: shutdown 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 msgr2=0x7f4ddc14b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 0x7f4ddc14b5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400eaa0 tx=0x7f4dc400ef70 comp rx=0 tx=0).stop 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 shutdown_connections 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4dac078e50 0x7f4dac07b310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4ddc1471a0 0x7f4ddc14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4ddc146cb0 0x7f4ddc14b5f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 --2- 10.20.193.18:0/2181291876 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4dc8004010 0x7f4ddc146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:04.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 >> 10.20.193.18:0/2181291876 conn(0x7f4ddc0cd780 msgr2=0x7f4ddc0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:04.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 shutdown_connections 2026-02-24T18:40:04.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 -- 10.20.193.18:0/2181291876 wait complete. 2026-02-24T18:40:04.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:04.702+0000 7f4de3238640 1 librados: shutdown 2026-02-24T18:40:07.714 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:07.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 10 monclient: get_monmap_and_config 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 10 monclient: build_initial_monmap 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 10 monclient: monmap: 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:07.784210+0000 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:07.784210+0000 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:07.785 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:07.786 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 1 Processor -- start 2026-02-24T18:40:07.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 1 -- start start 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.782+0000 7ff9b62b0640 10 monclient: init 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient: _add_conns ranks=[3,1,4,0,2,5] 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff9b006ff20 0x7ff9b0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-a con 0x7ff9b006ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff9b00c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7ff9b015a5e0 0x7ff9b014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-b con 0x7ff9b015a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b00c7e90 con 0x7ff9b00c99a0 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:07.789 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b0159a40 con 0x7ff9b006ff20 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00c8170 con 0x7ff9b015a5e0 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9b62b0640 10 monclient(hunting): authenticate will time out at 1751.465333s 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff9b006ff20 0x7ff9b0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff9b006ff20 0x7ff9b0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44434/0 (socket says 10.20.193.18:44434) 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 1 -- 10.20.193.18:0/4172625901 learned_addr learned my addr 10.20.193.18:0/4172625901 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 10 monclient(hunting): get_auth_request con 0x7ff9b006ff20 auth_method 0 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:07.790 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.783+0000 7ff9affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.784+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.784+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.784+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 10 monclient(hunting): handle_auth_done global_id 5109 payload 274 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 -- 10.20.193.18:0/4172625901 >> v2:10.20.193.32:3300/0 conn(0x7ff9b015a5e0 msgr2=0x7ff9b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 --2- 10.20.193.18:0/4172625901 >> v2:10.20.193.32:3300/0 conn(0x7ff9b015a5e0 0x7ff9b014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 -- 10.20.193.18:0/4172625901 >> v1:10.20.193.32:6789/0 conn(0x7ff9b00c99a0 legacy=0x7ff9b0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 monclient: found mon.noname-a 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:07.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:07.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 -- 10.20.193.18:0/4172625901 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00c8450 con 0x7ff9b006ff20 2026-02-24T18:40:07.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.785+0000 7ff9affff640 1 --2- 10.20.193.18:0/4172625901 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b0070300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009970 tx=0x7ff99c02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd22e93e29aa5c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:07.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 1 -- 10.20.193.18:0/4172625901 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03c070 con 0x7ff9b006ff20 2026-02-24T18:40:07.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: dump: 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:07.797 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 1 -- 10.20.193.18:0/4172625901 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c02fb20 con 0x7ff9b006ff20 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9b62b0640 5 monclient: authenticate success, global_id 5109 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9b62b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 1 -- 10.20.193.18:0/4172625901 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c037440 con 0x7ff9b006ff20 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9b62b0640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9b62b0640 4 set_mon_vals no callback set 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:07.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.787+0000 7ff9aeffd640 10 monclient: dump: 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 10 monclient: shutdown 2026-02-24T18:40:07.799 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- 10.20.193.18:0/4172625901 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 msgr2=0x7ff9b0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/4172625901 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b0070300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009970 tx=0x7ff99c02eef0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- 10.20.193.18:0/4172625901 shutdown_connections 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/4172625901 >> v2:10.20.193.32:3300/0 conn(0x7ff9b015a5e0 0x7ff9b014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/4172625901 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b0070300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- 10.20.193.18:0/4172625901 >> 10.20.193.18:0/4172625901 conn(0x7ff9b00cd780 msgr2=0x7ff9b00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- 10.20.193.18:0/4172625901 shutdown_connections 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- 10.20.193.18:0/4172625901 wait complete. 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 10 monclient: build_initial_monmap 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 10 monclient: monmap: 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:07.800 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 librados: starting msgr at 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 librados: starting objecter 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 Processor -- start 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 -- start start 2026-02-24T18:40:07.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 librados: setting wanted keys 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 1 librados: calling monclient init 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.788+0000 7ff9b62b0640 10 monclient: init 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-a con 0x7ff9b006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff9b00c99a0 0x7ff9b00bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-b con 0x7ff9b00c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff9b015a5e0 0x7ff9b00c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): picked mon.noname-c con 0x7ff9b015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0150350 con 0x7ff9b006ff20 2026-02-24T18:40:07.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00c8a10 con 0x7ff9b00c99a0 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01504d0 con 0x7ff9b015a5e0 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b62b0640 10 monclient(hunting): authenticate will time out at 1751.471295s 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44446/0 (socket says 10.20.193.18:44446) 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 1 -- 10.20.193.18:0/2271242819 learned_addr learned my addr 10.20.193.18:0/2271242819 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): get_auth_request con 0x7ff9b006ff20 auth_method 0 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:07.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b4826640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff9b015a5e0 0x7ff9b00c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.789+0000 7ff9b4826640 10 monclient(hunting): get_auth_request con 0x7ff9b015a5e0 auth_method 0 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff9b00c99a0 0x7ff9b00bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:07.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9b4826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): get_auth_request con 0x7ff9b00c99a0 auth_method 0 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.790+0000 7ff9af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 10 monclient(hunting): handle_auth_done global_id 5112 payload 824 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 -- 10.20.193.18:0/2271242819 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff9b015a5e0 msgr2=0x7ff9b00c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff9b015a5e0 0x7ff9b00c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 -- 10.20.193.18:0/2271242819 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff9b00c99a0 msgr2=0x7ff9b00bfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff9b00c99a0 0x7ff9b00bfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 monclient: found mon.noname-a 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:07.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.791+0000 7ff9affff640 1 -- 10.20.193.18:0/2271242819 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00be640 con 0x7ff9b006ff20 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.798+0000 7ff9affff640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009970 tx=0x7ff99c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.798+0000 7ff9acff9640 10 client.?.objecter ms_handle_connect 0x7ff9b006ff20 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.798+0000 7ff9acff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c044070 con 0x7ff9b006ff20 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: dump: 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:07.808 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:07.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c040610 con 0x7ff9b006ff20 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 5 monclient: authenticate success, global_id 5112 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _renew_subs 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b01f5a60 con 0x7ff9b006ff20 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03c040 con 0x7ff9b006ff20 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9ae7fc640 4 set_mon_vals no callback set 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9acff9640 10 monclient: dump: 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:07.810 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 client.5112.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _renew_subs 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01f1a60 con 0x7ff9b006ff20 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 1 librados: init done 2026-02-24T18:40:07.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:07.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:07.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:07.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.802+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b01508a0 con 0x7ff9b006ff20 2026-02-24T18:40:07.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.813+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff99c03f050 con 0x7ff9b006ff20 2026-02-24T18:40:07.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.814+0000 7ff9acff9640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 0x7ff98007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:07.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.814+0000 7ff9af7fe640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 0x7ff98007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:07.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.814+0000 7ff9af7fe640 10 monclient: get_auth_request con 0x7ff980078e50 auth_method 0 2026-02-24T18:40:07.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.814+0000 7ff9acff9640 10 client.5112.objecter ms_handle_connect 0x7ff980078e50 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.814+0000 7ff9af7fe640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 0x7ff98007b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00cc910 tx=0x7ff9a00091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.815+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7ff99c082df0 con 0x7ff9b006ff20 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.815+0000 7ff9acff9640 10 client.5112.objecter ms_dispatch2 0x7ff9b0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.815+0000 7ff9acff9640 3 client.5112.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.815+0000 7ff9acff9640 3 client.5112.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:07.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.815+0000 7ff9acff9640 20 client.5112.objecter dump_active .. 0 homeless 2026-02-24T18:40:07.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.914+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff99c086db0 con 0x7ff9b006ff20 2026-02-24T18:40:07.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.914+0000 7ff9acff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:07.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:07.914+0000 7ff9acff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:08.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.015+0000 7ff9b62b0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:08.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.015+0000 7ff9b62b0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:08.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.015+0000 7ff9b62b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:08.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.015+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff9b014df70 con 0x7ff9b006ff20 2026-02-24T18:40:08.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.018+0000 7ff9acff9640 1 -- 10.20.193.18:0/2271242819 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff99c08b0b0 con 0x7ff9b006ff20 2026-02-24T18:40:08.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.018+0000 7ff9acff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:08.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.018+0000 7ff9acff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:08.021 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:08.021 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:08.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 10 librados: watch_flush enter 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 10 librados: watch_flush exit 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 20 client.5112.objecter shutdown clearing up homeless session... 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 10 client.5112.objecter successfully canceled tick 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 msgr2=0x7ff98007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 0x7ff98007b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00cc910 tx=0x7ff9a00091c0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 10 monclient: shutdown 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 msgr2=0x7ff9b00c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009970 tx=0x7ff99c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 shutdown_connections 2026-02-24T18:40:08.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff980078e50 0x7ff98007b310 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff9b00c99a0 0x7ff9b00bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff9b015a5e0 0x7ff9b00c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 --2- 10.20.193.18:0/2271242819 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff9b006ff20 0x7ff9b00c7060 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 >> 10.20.193.18:0/2271242819 conn(0x7ff9b00cd780 msgr2=0x7ff9b0159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 shutdown_connections 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 -- 10.20.193.18:0/2271242819 wait complete. 2026-02-24T18:40:08.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:08.019+0000 7ff9b62b0640 1 librados: shutdown 2026-02-24T18:40:11.031 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.099+0000 7f50b9458640 10 monclient: get_monmap_and_config 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.099+0000 7f50b9458640 10 monclient: build_initial_monmap 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.099+0000 7f50b9458640 10 monclient: monmap: 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:11.100671+0000 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:11.100671+0000 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:11.101 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 Processor -- start 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 -- start start 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient: init 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient: _add_conns ranks=[0,4,3,1,5,2] 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f50b41527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f50b4152bb0 0x7f50b40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-b con 0x7f50b4152bb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f50b40c5fc0 0x7f50b40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-a con 0x7f50b40c5fc0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b406e300 con 0x7f50b41527d0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f50b406f1f0 con 0x7f50b40c5fc0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f50b40c6470 con 0x7f50b4152bb0 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:11.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.100+0000 7f50b9458640 10 monclient(hunting): authenticate will time out at 1754.781782s 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f50b4152bb0 0x7f50b40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b37fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f50b40c5fc0 0x7f50b40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b2ffd640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f50b41527d0 0x7f50b406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37676/0 (socket says 10.20.193.18:37676) 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b2ffd640 1 -- 10.20.193.18:0/38954275 learned_addr learned my addr 10.20.193.18:0/38954275 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): get_auth_request con 0x7f50b4152bb0 auth_method 0 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b37fe640 10 monclient(hunting): get_auth_request con 0x7f50b40c5fc0 auth_method 0 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.101+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.103 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 10 monclient(hunting): handle_auth_done global_id 5071 payload 274 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 1 -- 10.20.193.18:0/38954275 >> v2:10.20.193.18:3300/0 conn(0x7f50b40c5fc0 msgr2=0x7f50b40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 1 --2- 10.20.193.18:0/38954275 >> v2:10.20.193.18:3300/0 conn(0x7f50b40c5fc0 0x7f50b40c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 1 -- 10.20.193.18:0/38954275 >> v1:10.20.193.18:6789/0 conn(0x7f50b41527d0 legacy=0x7f50b406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 1 monclient: found mon.noname-b 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.102+0000 7f50b27fc640 1 -- 10.20.193.18:0/38954275 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b406e5e0 con 0x7f50b4152bb0 2026-02-24T18:40:11.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.103+0000 7f50b27fc640 1 --2- 10.20.193.18:0/38954275 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b4152bb0 0x7f50b40c5a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50b40750e0 tx=0x7f509c02ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ddccf610007273 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 1 -- 10.20.193.18:0/38954275 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c03c070 con 0x7f50b4152bb0 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: dump: 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:11.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 5 monclient: authenticate success, global_id 5071 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 1 -- 10.20.193.18:0/38954275 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c02faf0 con 0x7f50b4152bb0 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 1 -- 10.20.193.18:0/38954275 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c037440 con 0x7f50b4152bb0 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b1ffb640 10 monclient: dump: 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:11.107 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 4 set_mon_vals no callback set 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 10 monclient: shutdown 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 -- 10.20.193.18:0/38954275 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b4152bb0 msgr2=0x7f50b40c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 --2- 10.20.193.18:0/38954275 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b4152bb0 0x7f50b40c5a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50b40750e0 tx=0x7f509c02ede0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 -- 10.20.193.18:0/38954275 shutdown_connections 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 --2- 10.20.193.18:0/38954275 >> v2:10.20.193.18:3300/0 conn(0x7f50b40c5fc0 0x7f50b40c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 --2- 10.20.193.18:0/38954275 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b4152bb0 0x7f50b40c5a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 -- 10.20.193.18:0/38954275 >> 10.20.193.18:0/38954275 conn(0x7f50b40cd780 msgr2=0x7f50b40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 -- 10.20.193.18:0/38954275 shutdown_connections 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.105+0000 7f50b9458640 1 -- 10.20.193.18:0/38954275 wait complete. 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient: build_initial_monmap 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient: monmap: 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 librados: starting msgr at 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 librados: starting objecter 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 Processor -- start 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 -- start start 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 librados: setting wanted keys 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 librados: calling monclient init 2026-02-24T18:40:11.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient: init 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50b4148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-c con 0x7f50b40c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-b con 0x7f50b41527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50b40c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): picked mon.noname-a con 0x7f50b4152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50b406ee80 con 0x7f50b4152bb0 2026-02-24T18:40:11.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50b406eba0 con 0x7f50b41527d0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50b40c93a0 con 0x7f50b40c5fc0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.106+0000 7f50b9458640 10 monclient(hunting): authenticate will time out at 1754.787761s 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50b40c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50b40c10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59984/0 (socket says 10.20.193.18:59984) 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 1 -- 10.20.193.18:0/3031408228 learned_addr learned my addr 10.20.193.18:0/3031408228 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): get_auth_request con 0x7f50b4152bb0 auth_method 0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): get_auth_request con 0x7f50b41527d0 auth_method 0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50b4148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): get_auth_request con 0x7f50b40c5fc0 auth_method 0 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.107+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 10 monclient(hunting): handle_auth_done global_id 5074 payload 824 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 msgr2=0x7f50b4148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50b4148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 msgr2=0x7f50b40c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50b40c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 monclient: found mon.noname-b 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.108+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b40c1c10 con 0x7f50b41527d0 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.109+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 msgr2=0x7f50b41f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.109+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 msgr2=0x7f50b41f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.109+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f50b406f370 tx=0x7f509c0047c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-24T18:40:11.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.109+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-24T18:40:11.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.315 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50b27fc640 10 monclient: get_auth_request con 0x7f50b41527d0 auth_method 0 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50b27fc640 0 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient: ms_handle_reset current mon [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 msgr2=0x7f50b41f49d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f50b41527d0 0x7f50b41f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50800055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): picked mon.noname-c con 0x7f50b40c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:11.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50800080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): picked mon.noname-a con 0x7f50b4152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5080008b00 0x7f508000af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): picked mon.noname-b con 0x7f5080008b00 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f508000ba90 con 0x7f50b4152bb0 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f508000bc10 con 0x7f5080008b00 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f508000be10 con 0x7f50b40c5fc0 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.310+0000 7f50937fe640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50800080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): get_auth_request con 0x7f50b4152bb0 auth_method 0 2026-02-24T18:40:11.317 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50800055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5080008b00 0x7f508000af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): get_auth_request con 0x7f50b40c5fc0 auth_method 0 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): get_auth_request con 0x7f5080008b00 auth_method 0 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.311+0000 7f50b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 10 monclient(hunting): handle_auth_done global_id 5074 payload 293 2026-02-24T18:40:11.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 msgr2=0x7f50800055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50800055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5080008b00 msgr2=0x7f508000af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5080008b00 0x7f508000af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 monclient: found mon.noname-a 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.312+0000 7f50b27fc640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508000c0d0 con 0x7f50b4152bb0 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.313+0000 7f50b27fc640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50800080a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50b406f370 tx=0x7f509c033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.313+0000 7f50937fe640 10 client.?.objecter ms_handle_connect 0x7f50b4152bb0 2026-02-24T18:40:11.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.313+0000 7f50937fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:11.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c03c040 con 0x7f50b4152bb0 2026-02-24T18:40:11.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:11.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:11.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: dump: 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c042620 con 0x7f50b4152bb0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c042a10 con 0x7f50b4152bb0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50b9458640 5 monclient: authenticate success, global_id 5074 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50b9458640 10 monclient: _renew_subs 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50b17fa640 4 set_mon_vals no callback set 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50b9458640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b40c1ef0 con 0x7f50b4152bb0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:11.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.316+0000 7f50937fe640 10 monclient: dump: 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 client.5074.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:11.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 monclient: _renew_subs 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b40c24b0 con 0x7f50b4152bb0 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 1 librados: init done 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:11.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.317+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b40c9730 con 0x7f50b4152bb0 2026-02-24T18:40:11.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.329+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f509c03f040 con 0x7f50b4152bb0 2026-02-24T18:40:11.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.329+0000 7f50937fe640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 0x7f5080083350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:11.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.330+0000 7f50b2ffd640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 0x7f5080083350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:11.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.330+0000 7f50b2ffd640 10 monclient: get_auth_request con 0x7f50b41527d0 auth_method 0 2026-02-24T18:40:11.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.330+0000 7f50b2ffd640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 0x7f5080083350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50a8002770 tx=0x7f50a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:11.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.330+0000 7f50937fe640 10 client.5074.objecter ms_handle_connect 0x7f50b41527d0 2026-02-24T18:40:11.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f509c082600 con 0x7f50b4152bb0 2026-02-24T18:40:11.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 10 client.5074.objecter ms_dispatch2 0x7f50b4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 3 client.5074.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 3 client.5074.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 20 client.5074.objecter dump_active .. 0 homeless 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f509c054080 con 0x7f50b4152bb0 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:11.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.433+0000 7f50937fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:11.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.536+0000 7f50b9458640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:11.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.536+0000 7f50b9458640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:11.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.536+0000 7f50b9458640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:11.538 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.536+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f50b40c8ba0 con 0x7f50b4152bb0 2026-02-24T18:40:11.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.538+0000 7f50937fe640 1 -- 10.20.193.18:0/3031408228 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f509c0865c0 con 0x7f50b4152bb0 2026-02-24T18:40:11.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.538+0000 7f50937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:11.540 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.538+0000 7f50937fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:11.540 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:11.541 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:11.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 10 librados: watch_flush enter 2026-02-24T18:40:11.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 10 librados: watch_flush exit 2026-02-24T18:40:11.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 20 client.5074.objecter shutdown clearing up homeless session... 2026-02-24T18:40:11.542 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 10 client.5074.objecter successfully canceled tick 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 msgr2=0x7f5080083350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 0x7f5080083350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50a8002770 tx=0x7f50a8009040 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 10 monclient: shutdown 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 msgr2=0x7f50800080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.539+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50800080a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50b406f370 tx=0x7f509c033000 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 shutdown_connections 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f50b41527d0 0x7f5080083350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f5080008b00 0x7f508000af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f50b4152bb0 0x7f50800080a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 --2- 10.20.193.18:0/3031408228 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f50b40c5fc0 0x7f50800055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:11.543 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 >> 10.20.193.18:0/3031408228 conn(0x7f50b40cd780 msgr2=0x7f50b414fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:11.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 shutdown_connections 2026-02-24T18:40:11.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 -- 10.20.193.18:0/3031408228 wait complete. 2026-02-24T18:40:11.544 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:11.540+0000 7f50b9458640 1 librados: shutdown 2026-02-24T18:40:14.551 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:14.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 10 monclient: get_monmap_and_config 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 10 monclient: build_initial_monmap 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 10 monclient: monmap: 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:14.621031+0000 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:14.621031+0000 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:14.625 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 1 Processor -- start 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 1 -- start start 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.620+0000 7fdd286c6640 10 monclient: init 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient: _add_conns ranks=[5,4,3,1,2,0] 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fdd2006ff20 0x7fdd20070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-c con 0x7fdd2006ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fdd200c99a0 0x7fdd20159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-b con 0x7fdd200c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fdd2015a410 0x7fdd2014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-a con 0x7fdd2015a410 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fdd20159a40 con 0x7fdd2015a410 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2015a860 con 0x7fdd200c99a0 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdd200c7e90 con 0x7fdd2006ff20 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:14.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd286c6640 10 monclient(hunting): authenticate will time out at 1758.302135s 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fdd2015a410 0x7fdd2014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd25c3a640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fdd200c99a0 0x7fdd20159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd2643b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fdd2006ff20 0x7fdd20070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fdd2015a410 0x7fdd2014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:59994/0 (socket says 10.20.193.18:59994) 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 1 -- 10.20.193.18:0/2450389063 learned_addr learned my addr 10.20.193.18:0/2450389063 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 10 monclient(hunting): get_auth_request con 0x7fdd2015a410 auth_method 0 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd26c3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd25c3a640 10 monclient(hunting): get_auth_request con 0x7fdd200c99a0 auth_method 0 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd25c3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd25c3a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd25c3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd2643b640 10 monclient(hunting): get_auth_request con 0x7fdd2006ff20 auth_method 0 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd2643b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd2643b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.621+0000 7fdd2643b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.622+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_done global_id 5124 payload 274 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 -- 10.20.193.18:0/2450389063 >> v2:10.20.193.110:3300/0 conn(0x7fdd2006ff20 msgr2=0x7fdd20070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 --2- 10.20.193.18:0/2450389063 >> v2:10.20.193.110:3300/0 conn(0x7fdd2006ff20 0x7fdd20070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 -- 10.20.193.18:0/2450389063 >> v2:10.20.193.32:3300/0 conn(0x7fdd200c99a0 msgr2=0x7fdd20159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 --2- 10.20.193.18:0/2450389063 >> v2:10.20.193.32:3300/0 conn(0x7fdd200c99a0 0x7fdd20159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 10 monclient: _finish_hunting 0 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 monclient: found mon.noname-a 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:14.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.623+0000 7fdd26c3c640 1 -- 10.20.193.18:0/2450389063 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd200c8170 con 0x7fdd2015a410 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.624+0000 7fdd26c3c640 1 --2- 10.20.193.18:0/2450389063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd2015a410 0x7fdd2014fe10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd200709c0 tx=0x7fdd1c02f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6328f1f29e0ab9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 1 -- 10.20.193.18:0/2450389063 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c02fce0 con 0x7fdd2015a410 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: dump: 2026-02-24T18:40:14.637 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: _finish_auth 0 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: _check_auth_tickets 2026-02-24T18:40:14.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 1 -- 10.20.193.18:0/2450389063 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0373f0 con 0x7fdd2015a410 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 1 -- 10.20.193.18:0/2450389063 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0377c0 con 0x7fdd2015a410 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 5 monclient: authenticate success, global_id 5124 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 4 set_mon_vals no callback set 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd25439640 10 monclient: dump: 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:14.639 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 10 monclient: shutdown 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 -- 10.20.193.18:0/2450389063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd2015a410 msgr2=0x7fdd2014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 --2- 10.20.193.18:0/2450389063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd2015a410 0x7fdd2014fe10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd200709c0 tx=0x7fdd1c02f270 comp rx=0 tx=0).stop 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 -- 10.20.193.18:0/2450389063 shutdown_connections 2026-02-24T18:40:14.640 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 --2- 10.20.193.18:0/2450389063 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd2015a410 0x7fdd2014fe10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 --2- 10.20.193.18:0/2450389063 >> v2:10.20.193.32:3300/0 conn(0x7fdd200c99a0 0x7fdd20159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 --2- 10.20.193.18:0/2450389063 >> v2:10.20.193.110:3300/0 conn(0x7fdd2006ff20 0x7fdd20070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 -- 10.20.193.18:0/2450389063 >> 10.20.193.18:0/2450389063 conn(0x7fdd200cd780 msgr2=0x7fdd200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 -- 10.20.193.18:0/2450389063 shutdown_connections 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 1 -- 10.20.193.18:0/2450389063 wait complete. 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 10 monclient: build_initial_monmap 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.626+0000 7fdd286c6640 10 monclient: monmap: 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:14.641 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 librados: starting msgr at 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 librados: starting objecter 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 Processor -- start 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 -- start start 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 librados: setting wanted keys 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 librados: calling monclient init 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient: init 2026-02-24T18:40:14.642 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 0x7fdd201f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-c con 0x7fdd2006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 0x7fdd200c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-a con 0x7fdd200c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdd2015a410 0x7fdd200c0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): picked mon.noname-b con 0x7fdd2015a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd200c8a10 con 0x7fdd200c99a0 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd200c8730 con 0x7fdd2015a410 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:14.643 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20150350 con 0x7fdd2006ff20 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd286c6640 10 monclient(hunting): authenticate will time out at 1758.308708s 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.627+0000 7fdd25c3a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 0x7fdd200c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 0x7fdd201f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdd2015a410 0x7fdd200c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 0x7fdd200c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:60000/0 (socket says 10.20.193.18:60000) 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 1 -- 10.20.193.18:0/872861912 learned_addr learned my addr 10.20.193.18:0/872861912 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): get_auth_request con 0x7fdd200c99a0 auth_method 0 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): get_auth_request con 0x7fdd2006ff20 auth_method 0 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): get_auth_request con 0x7fdd2015a410 auth_method 0 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd25c3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd2643b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.628+0000 7fdd26c3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 10 monclient(hunting): handle_auth_done global_id 5063 payload 824 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 -- 10.20.193.18:0/872861912 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdd2015a410 msgr2=0x7fdd200c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdd2015a410 0x7fdd200c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 -- 10.20.193.18:0/872861912 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 msgr2=0x7fdd200c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 0x7fdd200c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 monclient: found mon.noname-c 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:14.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.629+0000 7fdd2643b640 1 -- 10.20.193.18:0/872861912 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd200c22c0 con 0x7fdd2006ff20 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.630+0000 7fdd2643b640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 0x7fdd201f24f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9e0 tx=0x7fdd1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.630+0000 7fdd06ffd640 10 client.?.objecter ms_handle_connect 0x7fdd2006ff20 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.630+0000 7fdd06ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10017070 con 0x7fdd2006ff20 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: dump: 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:14.647 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 5 monclient: authenticate success, global_id 5063 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _renew_subs 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd200c25a0 con 0x7fdd2006ff20 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 client.5063.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:14.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _renew_subs 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd200c2b60 con 0x7fdd2006ff20 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 1 librados: init done 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1000ca00 con 0x7fdd2006ff20 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10021060 con 0x7fdd2006ff20 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd24c38640 4 set_mon_vals no callback set 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd06ffd640 10 monclient: dump: 2026-02-24T18:40:14.649 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:14.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:14.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:14.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.632+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd201506e0 con 0x7fdd2006ff20 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.643+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fdd10013470 con 0x7fdd2006ff20 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.644+0000 7fdd06ffd640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 0x7fdcf007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.644+0000 7fdd25c3a640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 0x7fdcf007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.644+0000 7fdd25c3a640 10 monclient: get_auth_request con 0x7fdcf0078e00 auth_method 0 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.644+0000 7fdd06ffd640 10 client.5063.objecter ms_handle_connect 0x7fdcf0078e00 2026-02-24T18:40:14.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.644+0000 7fdd25c3a640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 0x7fdcf007b2c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd200cdcd0 tx=0x7fdd14009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:14.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.645+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fdd1000cdb0 con 0x7fdd2006ff20 2026-02-24T18:40:14.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.645+0000 7fdd06ffd640 10 client.5063.objecter ms_dispatch2 0x7fdd20000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:14.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.645+0000 7fdd06ffd640 3 client.5063.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:14.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.645+0000 7fdd06ffd640 3 client.5063.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:14.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.645+0000 7fdd06ffd640 20 client.5063.objecter dump_active .. 0 homeless 2026-02-24T18:40:14.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.744+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdd10062740 con 0x7fdd2006ff20 2026-02-24T18:40:14.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.744+0000 7fdd06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:14.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.744+0000 7fdd06ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:14.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.845+0000 7fdd286c6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:14.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.845+0000 7fdd286c6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:14.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.845+0000 7fdd286c6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:14.848 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.845+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd20157a90 con 0x7fdd2006ff20 2026-02-24T18:40:14.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.848+0000 7fdd06ffd640 1 -- 10.20.193.18:0/872861912 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fdd10066a40 con 0x7fdd2006ff20 2026-02-24T18:40:14.849 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.848+0000 7fdd06ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:14.850 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.848+0000 7fdd06ffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:14.850 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:14.850 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 10 librados: watch_flush enter 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 10 librados: watch_flush exit 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 20 client.5063.objecter shutdown clearing up homeless session... 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 10 client.5063.objecter successfully canceled tick 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 msgr2=0x7fdcf007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 0x7fdcf007b2c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd200cdcd0 tx=0x7fdd14009290 comp rx=0 tx=0).stop 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 10 monclient: shutdown 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 msgr2=0x7fdd201f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:14.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 0x7fdd201f24f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9e0 tx=0x7fdd1000eeb0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 shutdown_connections 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fdcf0078e00 0x7fdcf007b2c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fdd2015a410 0x7fdd200c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fdd200c99a0 0x7fdd200c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 --2- 10.20.193.18:0/872861912 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fdd2006ff20 0x7fdd201f24f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 >> 10.20.193.18:0/872861912 conn(0x7fdd200cd780 msgr2=0x7fdd2014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 shutdown_connections 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 -- 10.20.193.18:0/872861912 wait complete. 2026-02-24T18:40:14.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:14.849+0000 7fdd286c6640 1 librados: shutdown 2026-02-24T18:40:17.860 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:17.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.927+0000 7f0d671a9640 10 monclient: get_monmap_and_config 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.927+0000 7f0d671a9640 10 monclient: build_initial_monmap 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.927+0000 7f0d671a9640 10 monclient: monmap: 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:17.928547+0000 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:17.928547+0000 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:17.930 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 Processor -- start 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 -- start start 2026-02-24T18:40:17.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient: init 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient: _add_conns ranks=[4,3,0,5,1,2] 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0d6006ff20 0x7f0d60070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0d6006ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0d600c99a0 0x7f0d60159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-a con 0x7f0d600c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:17.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0d6015a410 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d6015a860 con 0x7f0d6015a410 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f0d60159a40 con 0x7f0d600c99a0 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600c7e90 con 0x7f0d6006ff20 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d671a9640 10 monclient(hunting): authenticate will time out at 1761.609679s 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.928+0000 7f0d57fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f0d600c99a0 0x7f0d60159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f0d6006ff20 0x7f0d60070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d6571f640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f0d6015a410 0x7f0d6014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:37686/0 (socket says 10.20.193.18:37686) 2026-02-24T18:40:17.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d6571f640 1 -- 10.20.193.18:0/3911275105 learned_addr learned my addr 10.20.193.18:0/3911275105 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): get_auth_request con 0x7f0d600c99a0 auth_method 0 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): get_auth_request con 0x7f0d6006ff20 auth_method 0 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:17.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.929+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_done global_id 5083 payload 274 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 1 -- 10.20.193.18:0/3911275105 >> v2:10.20.193.18:3300/0 conn(0x7f0d600c99a0 msgr2=0x7f0d60159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 1 --2- 10.20.193.18:0/3911275105 >> v2:10.20.193.18:3300/0 conn(0x7f0d600c99a0 0x7f0d60159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 1 -- 10.20.193.18:0/3911275105 >> v1:10.20.193.18:6789/0 conn(0x7f0d6015a410 legacy=0x7f0d6014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 10 monclient: _finish_hunting 0 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 1 monclient: found mon.noname-b 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.930+0000 7f0d64f1e640 1 -- 10.20.193.18:0/3911275105 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d600c8170 con 0x7f0d6006ff20 2026-02-24T18:40:17.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.931+0000 7f0d64f1e640 1 --2- 10.20.193.18:0/3911275105 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6006ff20 0x7f0d60070300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098a0 tx=0x7f0d5002ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afcf62abf1734b3f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 1 -- 10.20.193.18:0/3911275105 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5003c070 con 0x7f0d6006ff20 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: dump: 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.944 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 1 -- 10.20.193.18:0/3911275105 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5002fae0 con 0x7f0d6006ff20 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 5 monclient: authenticate success, global_id 5083 2026-02-24T18:40:17.945 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 1 -- 10.20.193.18:0/3911275105 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50037440 con 0x7f0d6006ff20 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d577fe640 10 monclient: dump: 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.946 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 4 set_mon_vals no callback set 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 10 monclient: shutdown 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 1 -- 10.20.193.18:0/3911275105 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6006ff20 msgr2=0x7f0d60070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.933+0000 7f0d671a9640 1 --2- 10.20.193.18:0/3911275105 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6006ff20 0x7f0d60070300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098a0 tx=0x7f0d5002ee10 comp rx=0 tx=0).stop 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 -- 10.20.193.18:0/3911275105 shutdown_connections 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 --2- 10.20.193.18:0/3911275105 >> v2:10.20.193.18:3300/0 conn(0x7f0d600c99a0 0x7f0d60159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 --2- 10.20.193.18:0/3911275105 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6006ff20 0x7f0d60070300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:17.947 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 -- 10.20.193.18:0/3911275105 >> 10.20.193.18:0/3911275105 conn(0x7f0d600cd780 msgr2=0x7f0d600cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 -- 10.20.193.18:0/3911275105 shutdown_connections 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 -- 10.20.193.18:0/3911275105 wait complete. 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 10 monclient: build_initial_monmap 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 10 monclient: monmap: 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:17.948 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 librados: starting msgr at 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 librados: starting objecter 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 Processor -- start 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 -- start start 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 librados: setting wanted keys 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 1 librados: calling monclient init 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.934+0000 7f0d671a9640 10 monclient: init 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d6006ff20 0x7f0d6014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-c con 0x7f0d6006ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d600c99a0 0x7f0d601f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-a con 0x7f0d600c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): picked mon.noname-b con 0x7f0d6015a410 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600c8a10 con 0x7f0d600c99a0 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600c8730 con 0x7f0d6015a410 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60150520 con 0x7f0d6006ff20 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:17.950 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d671a9640 10 monclient(hunting): authenticate will time out at 1761.616322s 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d57fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d600c99a0 0x7f0d601f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:54796/0 (socket says 10.20.193.18:54796) 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 1 -- 10.20.193.18:0/1803185220 learned_addr learned my addr 10.20.193.18:0/1803185220 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 10 monclient(hunting): get_auth_request con 0x7f0d6015a410 auth_method 0 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d6571f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d57fff640 10 monclient(hunting): get_auth_request con 0x7f0d600c99a0 auth_method 0 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d57fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:17.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.935+0000 7f0d64f1e640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d6006ff20 0x7f0d6014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d6571f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d6571f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d6571f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): get_auth_request con 0x7f0d6006ff20 auth_method 0 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:17.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.936+0000 7f0d64f1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 10 monclient(hunting): handle_auth_done global_id 5086 payload 824 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 -- 10.20.193.18:0/1803185220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d6006ff20 msgr2=0x7f0d6014d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d6006ff20 0x7f0d6014d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 -- 10.20.193.18:0/1803185220 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d600c99a0 msgr2=0x7f0d601f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d600c99a0 0x7f0d601f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 10 monclient: _finish_hunting 0 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 monclient: found mon.noname-b 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.937+0000 7f0d6571f640 1 -- 10.20.193.18:0/1803185220 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d60156430 con 0x7f0d6015a410 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.938+0000 7f0d6571f640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00cb70 tx=0x7f0d5c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:17.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.938+0000 7f0d557fa640 10 client.?.objecter ms_handle_connect 0x7f0d6015a410 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.938+0000 7f0d557fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d6015a410 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: dump: 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.954 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: _finish_auth 0 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: _check_auth_tickets 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00acf0 con 0x7f0d6015a410 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.939+0000 7f0d557fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c012600 con 0x7f0d6015a410 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 5 monclient: authenticate success, global_id 5086 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _renew_subs 2026-02-24T18:40:17.955 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d60156710 con 0x7f0d6015a410 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d56ffd640 4 set_mon_vals no callback set 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d557fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d557fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d557fa640 10 monclient: dump: 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:17.956 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 client.5086.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _renew_subs 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d60156cd0 con 0x7f0d6015a410 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 1 librados: init done 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:17.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.940+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d60070300 con 0x7f0d6015a410 2026-02-24T18:40:17.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.993+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0d5c01b3b0 con 0x7f0d6015a410 2026-02-24T18:40:17.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.993+0000 7f0d557fa640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 0x7f0d3007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:17.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.994+0000 7f0d64f1e640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 0x7f0d3007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:17.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.994+0000 7f0d64f1e640 10 monclient: get_auth_request con 0x7f0d30078e00 auth_method 0 2026-02-24T18:40:17.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.994+0000 7f0d64f1e640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 0x7f0d3007b2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d50039000 tx=0x7f0d50033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:17.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:17.994+0000 7f0d557fa640 10 client.5086.objecter ms_handle_connect 0x7f0d30078e00 2026-02-24T18:40:18.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.047+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f0d5c05dd20 con 0x7f0d6015a410 2026-02-24T18:40:18.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.047+0000 7f0d557fa640 10 client.5086.objecter ms_dispatch2 0x7f0d60000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:18.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.047+0000 7f0d557fa640 3 client.5086.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:18.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.047+0000 7f0d557fa640 3 client.5086.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:18.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.047+0000 7f0d557fa640 20 client.5086.objecter dump_active .. 0 homeless 2026-02-24T18:40:18.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.053+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0d5c061ce0 con 0x7f0d6015a410 2026-02-24T18:40:18.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.053+0000 7f0d557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:18.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.053+0000 7f0d557fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:18.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.155+0000 7f0d671a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:18.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.155+0000 7f0d671a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:18.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.155+0000 7f0d671a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:18.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.155+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0d601587f0 con 0x7f0d6015a410 2026-02-24T18:40:18.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.157+0000 7f0d557fa640 1 -- 10.20.193.18:0/1803185220 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0d5c065fe0 con 0x7f0d6015a410 2026-02-24T18:40:18.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.157+0000 7f0d557fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:18.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.157+0000 7f0d557fa640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:18.159 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:18.160 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 10 librados: watch_flush enter 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 10 librados: watch_flush exit 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 20 client.5086.objecter shutdown clearing up homeless session... 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 10 client.5086.objecter successfully canceled tick 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 msgr2=0x7f0d3007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:18.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 0x7f0d3007b2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d50039000 tx=0x7f0d50033000 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 10 monclient: shutdown 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 msgr2=0x7f0d601f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00cb70 tx=0x7f0d5c00a430 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 shutdown_connections 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0d30078e00 0x7f0d3007b2c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f0d6015a410 0x7f0d601f53d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f0d600c99a0 0x7f0d601f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 --2- 10.20.193.18:0/1803185220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f0d6006ff20 0x7f0d6014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 >> 10.20.193.18:0/1803185220 conn(0x7f0d600cd780 msgr2=0x7f0d6014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 shutdown_connections 2026-02-24T18:40:18.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.158+0000 7f0d671a9640 1 -- 10.20.193.18:0/1803185220 wait complete. 2026-02-24T18:40:18.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:18.159+0000 7f0d671a9640 1 librados: shutdown 2026-02-24T18:40:19.718 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:57.664 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.720 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:57.665 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.727 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:06:57.673 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.747 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:57.655 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.748 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:57.656 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.748 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:57.694 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.749 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:57.695 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.756 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:06:57.664 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:19.756 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:06:57.702 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:21.171 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:21.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.239+0000 7f7f42416640 10 monclient: get_monmap_and_config 2026-02-24T18:40:21.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.239+0000 7f7f42416640 10 monclient: build_initial_monmap 2026-02-24T18:40:21.241 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.239+0000 7f7f42416640 10 monclient: monmap: 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:21.240788+0000 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:21.240788+0000 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:21.242 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:21.243 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 Processor -- start 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 -- start start 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient: init 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient: _add_conns ranks=[2,1,3,4,0,5] 2026-02-24T18:40:21.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7f3c06ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7f3c0c99a0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7f3c15a5e0 0x7f7f3c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-a con 0x7f7f3c15a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0c7e90 con 0x7f7f3c0c99a0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0c8170 con 0x7f7f3c06ff20 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3c159a40 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.240+0000 7f7f42416640 10 monclient(hunting): authenticate will time out at 1764.921925s 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7f3c15a5e0 0x7f7f3c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f3bfff640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f7f3c06ff20 0x7f7f3c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:50560/0 (socket says 10.20.193.18:50560) 2026-02-24T18:40:21.246 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f3bfff640 1 -- 10.20.193.18:0/396003499 learned_addr learned my addr 10.20.193.18:0/396003499 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): get_auth_request con 0x7f7f3c15a5e0 auth_method 0 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.241+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 10 monclient(hunting): handle_auth_done global_id 5139 payload 274 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 1 -- 10.20.193.18:0/396003499 >> v1:10.20.193.110:6789/0 conn(0x7f7f3c06ff20 legacy=0x7f7f3c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 1 -- 10.20.193.18:0/396003499 >> v1:10.20.193.32:6789/0 conn(0x7f7f3c0c99a0 legacy=0x7f7f3c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 10 monclient: _finish_hunting 0 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 1 monclient: found mon.noname-a 2026-02-24T18:40:21.247 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:21.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:21.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.243+0000 7f7f4098c640 1 -- 10.20.193.18:0/396003499 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c0c8450 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.248 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.244+0000 7f7f4098c640 1 --2- 10.20.193.18:0/396003499 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c070300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009dc0 tx=0x7f7f3002f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6816062a5b86f926 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 1 -- 10.20.193.18:0/396003499 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3003c070 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: dump: 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.253 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: _finish_auth 0 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 1 -- 10.20.193.18:0/396003499 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f30037440 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f42416640 5 monclient: authenticate success, global_id 5139 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f42416640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:21.254 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 1 -- 10.20.193.18:0/396003499 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30037810 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f3affd640 10 monclient: dump: 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:21.255 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f42416640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.245+0000 7f7f42416640 4 set_mon_vals no callback set 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 10 monclient: shutdown 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 -- 10.20.193.18:0/396003499 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 msgr2=0x7f7f3c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 --2- 10.20.193.18:0/396003499 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c070300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009dc0 tx=0x7f7f3002f280 comp rx=0 tx=0).stop 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 -- 10.20.193.18:0/396003499 shutdown_connections 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 --2- 10.20.193.18:0/396003499 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c070300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 -- 10.20.193.18:0/396003499 >> 10.20.193.18:0/396003499 conn(0x7f7f3c0cd780 msgr2=0x7f7f3c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 -- 10.20.193.18:0/396003499 shutdown_connections 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 1 -- 10.20.193.18:0/396003499 wait complete. 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 10 monclient: build_initial_monmap 2026-02-24T18:40:21.256 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.246+0000 7f7f42416640 10 monclient: monmap: 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 librados: starting msgr at 2026-02-24T18:40:21.257 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 librados: starting objecter 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 Processor -- start 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 -- start start 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 librados: setting wanted keys 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 librados: calling monclient init 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient: init 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 0x7f7f3c157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-b con 0x7f7f3c06ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7f3c0c99a0 0x7f7f3c0c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-c con 0x7f7f3c0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:21.258 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): picked mon.noname-a con 0x7f7f3c15a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c150520 con 0x7f7f3c15a5e0 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c0c8a10 con 0x7f7f3c06ff20 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1506a0 con 0x7f7f3c0c99a0 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f42416640 10 monclient(hunting): authenticate will time out at 1764.928567s 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f4098c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c0c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.247+0000 7f7f3bfff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 0x7f7f3c157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7f3c0c99a0 0x7f7f3c0c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c0c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44220/0 (socket says 10.20.193.18:44220) 2026-02-24T18:40:21.259 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 1 -- 10.20.193.18:0/253780357 learned_addr learned my addr 10.20.193.18:0/253780357 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): get_auth_request con 0x7f7f3c15a5e0 auth_method 0 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): get_auth_request con 0x7f7f3c06ff20 auth_method 0 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): get_auth_request con 0x7f7f3c0c99a0 auth_method 0 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:21.260 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f4098c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.248+0000 7f7f3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 10 monclient(hunting): handle_auth_done global_id 5092 payload 824 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 -- 10.20.193.18:0/253780357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7f3c0c99a0 msgr2=0x7f7f3c0c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7f3c0c99a0 0x7f7f3c0c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 -- 10.20.193.18:0/253780357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 msgr2=0x7f7f3c0c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.261 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c0c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 monclient: found mon.noname-b 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.249+0000 7f7f3bfff640 1 -- 10.20.193.18:0/253780357 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c06dec0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.250+0000 7f7f3bfff640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 0x7f7f3c157a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800c500 tx=0x7f7f2800c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.250+0000 7f7f38ff9640 10 client.?.objecter ms_handle_connect 0x7f7f3c06ff20 2026-02-24T18:40:21.262 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.250+0000 7f7f38ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28017070 con 0x7f7f3c06ff20 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 10 monclient: dump: 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.264 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 5 monclient: authenticate success, global_id 5092 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 10 monclient: _renew_subs 2026-02-24T18:40:21.265 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c06d060 con 0x7f7f3c06ff20 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 10 client.5092.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 10 monclient: _renew_subs 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c06d620 con 0x7f7f3c06ff20 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.251+0000 7f7f42416640 1 librados: init done 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f42416640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f42416640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f42416640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c070300 con 0x7f7f3c06ff20 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f28005de0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:21.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28021060 con 0x7f7f3c06ff20 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f38ff9640 10 monclient: dump: 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:21.267 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:21.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.252+0000 7f7f3a7fc640 4 set_mon_vals no callback set 2026-02-24T18:40:21.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.355+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7f28024080 con 0x7f7f3c06ff20 2026-02-24T18:40:21.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.355+0000 7f7f38ff9640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 0x7f7f0c07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:21.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.355+0000 7f7f3b7fe640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 0x7f7f0c07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:21.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.355+0000 7f7f3b7fe640 10 monclient: get_auth_request con 0x7f7f0c078e30 auth_method 0 2026-02-24T18:40:21.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.356+0000 7f7f3b7fe640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 0x7f7f0c07b2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c06e980 tx=0x7f7f2c007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:21.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.356+0000 7f7f38ff9640 10 client.5092.objecter ms_handle_connect 0x7f7f0c078e30 2026-02-24T18:40:21.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.357+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7f2805d3e0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.357+0000 7f7f38ff9640 10 client.5092.objecter ms_dispatch2 0x7f7f3c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:21.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.357+0000 7f7f38ff9640 3 client.5092.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:21.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.357+0000 7f7f38ff9640 3 client.5092.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:21.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.357+0000 7f7f38ff9640 20 client.5092.objecter dump_active .. 0 homeless 2026-02-24T18:40:21.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.363+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7f280613a0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.363+0000 7f7f38ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:21.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.363+0000 7f7f38ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:21.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.461+0000 7f7f42416640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:21.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.461+0000 7f7f42416640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:21.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.461+0000 7f7f42416640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:21.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.461+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7f3c0c2fd0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.464+0000 7f7f38ff9640 1 -- 10.20.193.18:0/253780357 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7f280656a0 con 0x7f7f3c06ff20 2026-02-24T18:40:21.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.464+0000 7f7f38ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:21.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.464+0000 7f7f38ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:21.466 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:21.466 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:21.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 10 librados: watch_flush enter 2026-02-24T18:40:21.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 10 librados: watch_flush exit 2026-02-24T18:40:21.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 20 client.5092.objecter shutdown clearing up homeless session... 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 10 client.5092.objecter successfully canceled tick 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 msgr2=0x7f7f0c07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 0x7f7f0c07b2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c06e980 tx=0x7f7f2c007f30 comp rx=0 tx=0).stop 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 10 monclient: shutdown 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 msgr2=0x7f7f3c157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 0x7f7f3c157a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800c500 tx=0x7f7f2800c9d0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 shutdown_connections 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7f0c078e30 0x7f7f0c07b2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7f3c15a5e0 0x7f7f3c0c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7f3c0c99a0 0x7f7f3c0c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 --2- 10.20.193.18:0/253780357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7f3c06ff20 0x7f7f3c157a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:21.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 >> 10.20.193.18:0/253780357 conn(0x7f7f3c0cd780 msgr2=0x7f7f3c158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:21.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 shutdown_connections 2026-02-24T18:40:21.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 -- 10.20.193.18:0/253780357 wait complete. 2026-02-24T18:40:21.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:21.465+0000 7f7f42416640 1 librados: shutdown 2026-02-24T18:40:24.476 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 10 monclient: get_monmap_and_config 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 10 monclient: build_initial_monmap 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 10 monclient: monmap: 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:24.545329+0000 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:24.545329+0000 2026-02-24T18:40:24.546 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:24.547 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 1 Processor -- start 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 1 -- start start 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.544+0000 7f702a82b640 10 monclient: init 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient: _add_conns ranks=[3,5,2,0,1,4] 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f70241527d0 0x7f702406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-a con 0x7f70241527d0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:24.550 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f7024152bb0 0x7f70240c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-c con 0x7f7024152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f70240c5fc0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702406e300 con 0x7f70240c5fc0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f702406f1f0 con 0x7f70241527d0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f70240c6470 con 0x7f7024152bb0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f702a82b640 10 monclient(hunting): authenticate will time out at 1768.226472s 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.545+0000 7f7023fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f70241527d0 0x7f702406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f7024152bb0 0x7f70240c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f70241527d0 0x7f702406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44228/0 (socket says 10.20.193.18:44228) 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 1 -- 10.20.193.18:0/3976494439 learned_addr learned my addr 10.20.193.18:0/3976494439 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): get_auth_request con 0x7f70241527d0 auth_method 0 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): get_auth_request con 0x7f7024152bb0 auth_method 0 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:24.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.546+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 10 monclient(hunting): handle_auth_done global_id 5145 payload 274 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 1 -- 10.20.193.18:0/3976494439 >> v2:10.20.193.110:3300/0 conn(0x7f7024152bb0 msgr2=0x7f70240c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 1 --2- 10.20.193.18:0/3976494439 >> v2:10.20.193.110:3300/0 conn(0x7f7024152bb0 0x7f70240c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 1 -- 10.20.193.18:0/3976494439 >> v1:10.20.193.110:6789/0 conn(0x7f70240c5fc0 legacy=0x7f70240c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 1 monclient: found mon.noname-a 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.547+0000 7f7023fff640 1 -- 10.20.193.18:0/3976494439 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702406e5e0 con 0x7f70241527d0 2026-02-24T18:40:24.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.548+0000 7f7023fff640 1 --2- 10.20.193.18:0/3976494439 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70241527d0 0x7f702406da60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f70240750e0 tx=0x7f701002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98fefca2b526982 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:24.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 1 -- 10.20.193.18:0/3976494439 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701003c070 con 0x7f70241527d0 2026-02-24T18:40:24.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:24.559 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: dump: 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.560 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 1 -- 10.20.193.18:0/3976494439 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701002fb40 con 0x7f70241527d0 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 5 monclient: authenticate success, global_id 5145 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 1 -- 10.20.193.18:0/3976494439 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010037440 con 0x7f70241527d0 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 4 set_mon_vals no callback set 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:24.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f7022ffd640 10 monclient: dump: 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 10 monclient: shutdown 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:24.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 -- 10.20.193.18:0/3976494439 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70241527d0 msgr2=0x7f702406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 --2- 10.20.193.18:0/3976494439 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70241527d0 0x7f702406da60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f70240750e0 tx=0x7f701002eed0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 -- 10.20.193.18:0/3976494439 shutdown_connections 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 --2- 10.20.193.18:0/3976494439 >> v2:10.20.193.110:3300/0 conn(0x7f7024152bb0 0x7f70240c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 --2- 10.20.193.18:0/3976494439 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70241527d0 0x7f702406da60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 -- 10.20.193.18:0/3976494439 >> 10.20.193.18:0/3976494439 conn(0x7f70240cd780 msgr2=0x7f70240cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 -- 10.20.193.18:0/3976494439 shutdown_connections 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 -- 10.20.193.18:0/3976494439 wait complete. 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 10 monclient: build_initial_monmap 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 10 monclient: monmap: 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:24.563 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 librados: starting msgr at 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.550+0000 7f702a82b640 1 librados: starting objecter 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 Processor -- start 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 -- start start 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 librados: setting wanted keys 2026-02-24T18:40:24.564 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 librados: calling monclient init 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient: init 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-a con 0x7f70240c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f70241527d0 0x7f7024073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-b con 0x7f70241527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7024152bb0 0x7f70241f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): picked mon.noname-c con 0x7f7024152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f702406ee80 con 0x7f70240c5fc0 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:24.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f702406eba0 con 0x7f70241527d0 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f702406dfa0 con 0x7f7024152bb0 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f702a82b640 10 monclient(hunting): authenticate will time out at 1768.232503s 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7028da1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7024152bb0 0x7f70241f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44232/0 (socket says 10.20.193.18:44232) 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 1 -- 10.20.193.18:0/3120880159 learned_addr learned my addr 10.20.193.18:0/3120880159 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 10 monclient(hunting): get_auth_request con 0x7f70240c5fc0 auth_method 0 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:24.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.551+0000 7f7023fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f70241527d0 0x7f7024073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): get_auth_request con 0x7f7024152bb0 auth_method 0 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7023fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): get_auth_request con 0x7f70241527d0 auth_method 0 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:24.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f7028da1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.552+0000 7f70237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 10 monclient(hunting): handle_auth_done global_id 5148 payload 824 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 -- 10.20.193.18:0/3120880159 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7024152bb0 msgr2=0x7f70241f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7024152bb0 0x7f70241f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 -- 10.20.193.18:0/3120880159 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f70241527d0 msgr2=0x7f7024073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f70241527d0 0x7f7024073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 monclient: found mon.noname-a 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:24.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.553+0000 7f7023fff640 1 -- 10.20.193.18:0/3120880159 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70240c2e10 con 0x7f70240c5fc0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.554+0000 7f7023fff640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7010009a70 tx=0x7f70100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.554+0000 7f7020ff9640 10 client.?.objecter ms_handle_connect 0x7f70240c5fc0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.554+0000 7f7020ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701003c040 con 0x7f70240c5fc0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: dump: 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.569 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7010042410 con 0x7f70240c5fc0 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 5 monclient: authenticate success, global_id 5148 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _renew_subs 2026-02-24T18:40:24.570 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70240c1550 con 0x7f70240c5fc0 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010042800 con 0x7f70240c5fc0 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f70227fc640 4 set_mon_vals no callback set 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f7020ff9640 10 monclient: dump: 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:24.571 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 client.5148.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _renew_subs 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70240c1b10 con 0x7f70240c5fc0 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 1 librados: init done 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.556+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702406da60 con 0x7f70240c5fc0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.567+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f70100429a0 con 0x7f70240c5fc0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.568+0000 7f7020ff9640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 0x7f6ff407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.568+0000 7f70237fe640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 0x7f6ff407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.568+0000 7f70237fe640 10 monclient: get_auth_request con 0x7f6ff4078e00 auth_method 0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.569+0000 7f70237fe640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 0x7f6ff407b2c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70241f1690 tx=0x7f7014009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.569+0000 7f7020ff9640 10 client.5148.objecter ms_handle_connect 0x7f6ff4078e00 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.570+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7010083750 con 0x7f70240c5fc0 2026-02-24T18:40:24.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.570+0000 7f7020ff9640 10 client.5148.objecter ms_dispatch2 0x7f7024000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:24.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.570+0000 7f7020ff9640 3 client.5148.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:24.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.570+0000 7f7020ff9640 3 client.5148.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:24.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.570+0000 7f7020ff9640 20 client.5148.objecter dump_active .. 0 homeless 2026-02-24T18:40:24.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.669+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7010087710 con 0x7f70240c5fc0 2026-02-24T18:40:24.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.669+0000 7f7020ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:24.671 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.669+0000 7f7020ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:24.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.772+0000 7f702a82b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:24.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.772+0000 7f702a82b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:24.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.772+0000 7f702a82b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:24.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.772+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f70240c89a0 con 0x7f70240c5fc0 2026-02-24T18:40:24.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.774+0000 7f7020ff9640 1 -- 10.20.193.18:0/3120880159 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f701008ba10 con 0x7f70240c5fc0 2026-02-24T18:40:24.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.774+0000 7f7020ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:24.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.774+0000 7f7020ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:24.776 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:24.776 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 10 librados: watch_flush enter 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 10 librados: watch_flush exit 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 20 client.5148.objecter shutdown clearing up homeless session... 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 10 client.5148.objecter successfully canceled tick 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 msgr2=0x7f6ff407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 0x7f6ff407b2c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70241f1690 tx=0x7f7014009290 comp rx=0 tx=0).stop 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 10 monclient: shutdown 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:24.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 msgr2=0x7f70240c28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7010009a70 tx=0x7f70100047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 shutdown_connections 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6ff4078e00 0x7f6ff407b2c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f70241527d0 0x7f7024073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7024152bb0 0x7f70241f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 --2- 10.20.193.18:0/3120880159 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f70240c5fc0 0x7f70240c28d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 >> 10.20.193.18:0/3120880159 conn(0x7f70240cd780 msgr2=0x7f70240cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 shutdown_connections 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 -- 10.20.193.18:0/3120880159 wait complete. 2026-02-24T18:40:24.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:24.775+0000 7f702a82b640 1 librados: shutdown 2026-02-24T18:40:27.787 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:27.859 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 10 monclient: get_monmap_and_config 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 10 monclient: build_initial_monmap 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 10 monclient: monmap: 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:27.859188+0000 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:27.859188+0000 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.860 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:27.861 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.863 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 1 Processor -- start 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 1 -- start start 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.858+0000 7ff12e6fa640 10 monclient: init 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient: _add_conns ranks=[3,2,5,1,0,4] 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7ff12806ff20 0x7ff128070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-a con 0x7ff12806ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff1280c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff12815a5e0 0x7ff12814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-c con 0x7ff12815a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1280c7e90 con 0x7ff1280c99a0 2026-02-24T18:40:27.864 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7ff128159a40 con 0x7ff12806ff20 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff1280c8170 con 0x7ff12815a5e0 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12e6fa640 10 monclient(hunting): authenticate will time out at 1771.540324s 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.859+0000 7ff12cc70640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff12815a5e0 0x7ff12814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7ff12815a5e0 0x7ff12814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:50740/0 (socket says 10.20.193.18:50740) 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 1 -- 10.20.193.18:0/1677106802 learned_addr learned my addr 10.20.193.18:0/1677106802 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): get_auth_request con 0x7ff12815a5e0 auth_method 0 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:27.865 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.860+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 10 monclient(hunting): handle_auth_done global_id 5081 payload 274 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 1 -- 10.20.193.18:0/1677106802 >> v2:10.20.193.18:3300/0 conn(0x7ff12806ff20 msgr2=0x7ff128070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 1 --2- 10.20.193.18:0/1677106802 >> v2:10.20.193.18:3300/0 conn(0x7ff12806ff20 0x7ff128070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 1 -- 10.20.193.18:0/1677106802 >> v1:10.20.193.110:6789/0 conn(0x7ff1280c99a0 legacy=0x7ff128159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 10 monclient: _finish_hunting 0 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 1 monclient: found mon.noname-c 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.861+0000 7ff12cc70640 1 -- 10.20.193.18:0/1677106802 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1280c8450 con 0x7ff12815a5e0 2026-02-24T18:40:27.866 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.862+0000 7ff12cc70640 1 --2- 10.20.193.18:0/1677106802 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12815a5e0 0x7ff12814fe10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009e50 tx=0x7ff11c02f2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889ece0cc15a4ccb server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:27.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 1 -- 10.20.193.18:0/1677106802 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c02fcc0 con 0x7ff12815a5e0 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: dump: 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:27.872 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 5 monclient: authenticate success, global_id 5081 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 1 -- 10.20.193.18:0/1677106802 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c0373f0 con 0x7ff12815a5e0 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 1 -- 10.20.193.18:0/1677106802 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c02fe60 con 0x7ff12815a5e0 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:27.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff126ffd640 10 monclient: dump: 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 4 set_mon_vals no callback set 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 10 monclient: shutdown 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 -- 10.20.193.18:0/1677106802 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12815a5e0 msgr2=0x7ff12814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/1677106802 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12815a5e0 0x7ff12814fe10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009e50 tx=0x7ff11c02f2a0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 -- 10.20.193.18:0/1677106802 shutdown_connections 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/1677106802 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12815a5e0 0x7ff12814fe10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/1677106802 >> v2:10.20.193.18:3300/0 conn(0x7ff12806ff20 0x7ff128070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 -- 10.20.193.18:0/1677106802 >> 10.20.193.18:0/1677106802 conn(0x7ff1280cd780 msgr2=0x7ff1280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 -- 10.20.193.18:0/1677106802 shutdown_connections 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.864+0000 7ff12e6fa640 1 -- 10.20.193.18:0/1677106802 wait complete. 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 10 monclient: build_initial_monmap 2026-02-24T18:40:27.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 10 monclient: monmap: 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 librados: starting msgr at 2026-02-24T18:40:27.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 librados: starting objecter 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 Processor -- start 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 -- start start 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 librados: setting wanted keys 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 1 librados: calling monclient init 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.865+0000 7ff12e6fa640 10 monclient: init 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 0x7ff128147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-c con 0x7ff12806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff1280c99a0 0x7ff1280ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-b con 0x7ff1280c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): picked mon.noname-a con 0x7ff12815a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff128150350 con 0x7ff12815a5e0 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280c8a10 con 0x7ff1280c99a0 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281504d0 con 0x7ff12806ff20 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12e6fa640 10 monclient(hunting): authenticate will time out at 1771.547305s 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 0x7ff128147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:44252/0 (socket says 10.20.193.18:44252) 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 1 -- 10.20.193.18:0/3769240357 learned_addr learned my addr 10.20.193.18:0/3769240357 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:27.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): get_auth_request con 0x7ff12815a5e0 auth_method 0 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): get_auth_request con 0x7ff12806ff20 auth_method 0 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff1277fe640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff1280c99a0 0x7ff1280ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff1277fe640 10 monclient(hunting): get_auth_request con 0x7ff1280c99a0 auth_method 0 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff1277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff1277fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff1277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:27.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff12cc70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.866+0000 7ff127fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.867+0000 7ff1277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.867+0000 7ff1277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.867+0000 7ff1277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 10 monclient(hunting): handle_auth_done global_id 5084 payload 824 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 -- 10.20.193.18:0/3769240357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff1280c99a0 msgr2=0x7ff1280ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff1280c99a0 0x7ff1280ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 -- 10.20.193.18:0/3769240357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 msgr2=0x7ff1281f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 monclient: found mon.noname-c 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff127fff640 1 -- 10.20.193.18:0/3769240357 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1280cc000 con 0x7ff12806ff20 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.868+0000 7ff12cc70640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.869+0000 7ff127fff640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 0x7ff128147320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b580 tx=0x7ff11400ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.869+0000 7ff124ff9640 10 client.?.objecter ms_handle_connect 0x7ff12806ff20 2026-02-24T18:40:27.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.869+0000 7ff124ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114004480 con 0x7ff12806ff20 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 10 monclient: dump: 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:27.883 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.870+0000 7ff124ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff114014070 con 0x7ff12806ff20 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114002920 con 0x7ff12806ff20 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 5 monclient: authenticate success, global_id 5084 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _renew_subs 2026-02-24T18:40:27.884 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff1267fc640 4 set_mon_vals no callback set 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1280cc2e0 con 0x7ff12806ff20 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff124ff9640 10 monclient: dump: 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:27.885 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 client.5084.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _renew_subs 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1280cc8a0 con 0x7ff12806ff20 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 1 librados: init done 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:27.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.871+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1281508a0 con 0x7ff12806ff20 2026-02-24T18:40:27.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.883+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff114002ac0 con 0x7ff12806ff20 2026-02-24T18:40:27.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.883+0000 7ff124ff9640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 0x7ff0f807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:27.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.883+0000 7ff1277fe640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 0x7ff0f807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:27.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.883+0000 7ff1277fe640 10 monclient: get_auth_request con 0x7ff0f8078e00 auth_method 0 2026-02-24T18:40:27.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.884+0000 7ff124ff9640 10 client.5084.objecter ms_handle_connect 0x7ff0f8078e00 2026-02-24T18:40:27.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.884+0000 7ff1277fe640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 0x7ff0f807b2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff12806f4e0 tx=0x7ff1180091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:27.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.976+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7ff11405dee0 con 0x7ff12806ff20 2026-02-24T18:40:27.977 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.976+0000 7ff124ff9640 10 client.5084.objecter ms_dispatch2 0x7ff128000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:27.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.976+0000 7ff124ff9640 3 client.5084.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:27.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.976+0000 7ff124ff9640 3 client.5084.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:27.978 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.976+0000 7ff124ff9640 20 client.5084.objecter dump_active .. 0 homeless 2026-02-24T18:40:27.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.983+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff114061ea0 con 0x7ff12806ff20 2026-02-24T18:40:27.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.983+0000 7ff124ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:27.984 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:27.983+0000 7ff124ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:28.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.083+0000 7ff12e6fa640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:28.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.084+0000 7ff12e6fa640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:28.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.084+0000 7ff12e6fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:28.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.084+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff12815a070 con 0x7ff12806ff20 2026-02-24T18:40:28.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.086+0000 7ff124ff9640 1 -- 10.20.193.18:0/3769240357 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff1140661a0 con 0x7ff12806ff20 2026-02-24T18:40:28.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.086+0000 7ff124ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:28.088 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.086+0000 7ff124ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:28.088 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:28.088 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 10 librados: watch_flush enter 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 10 librados: watch_flush exit 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 20 client.5084.objecter shutdown clearing up homeless session... 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 10 client.5084.objecter successfully canceled tick 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 msgr2=0x7ff0f807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 0x7ff0f807b2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff12806f4e0 tx=0x7ff1180091c0 comp rx=0 tx=0).stop 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 10 monclient: shutdown 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 msgr2=0x7ff128147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:28.090 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 0x7ff128147320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b580 tx=0x7ff11400ba50 comp rx=0 tx=0).stop 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 shutdown_connections 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7ff0f8078e00 0x7ff0f807b2c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7ff1280c99a0 0x7ff1280ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7ff12815a5e0 0x7ff1281f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 --2- 10.20.193.18:0/3769240357 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7ff12806ff20 0x7ff128147320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 >> 10.20.193.18:0/3769240357 conn(0x7ff1280cd780 msgr2=0x7ff1280c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 shutdown_connections 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 -- 10.20.193.18:0/3769240357 wait complete. 2026-02-24T18:40:28.091 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:28.087+0000 7ff12e6fa640 1 librados: shutdown 2026-02-24T18:40:31.099 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:31.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 10 monclient: get_monmap_and_config 2026-02-24T18:40:31.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 10 monclient: build_initial_monmap 2026-02-24T18:40:31.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 10 monclient: monmap: 2026-02-24T18:40:31.169 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:31.169 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:31.168224+0000 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:31.168224+0000 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 1 Processor -- start 2026-02-24T18:40:31.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 1 -- start start 2026-02-24T18:40:31.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.167+0000 7f07a6474640 10 monclient: init 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient: _add_conns ranks=[0,3,1,4,2,5] 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07a01527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f07a0152bb0 0x7f07a00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-a con 0x7f07a0152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f07a00c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a006e300 con 0x7f07a01527d0 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a006e5e0 con 0x7f07a00c5fc0 2026-02-24T18:40:31.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f07a006f1f0 con 0x7f07a0152bb0 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a6474640 10 monclient(hunting): authenticate will time out at 1774.849362s 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f079f7fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f07a0152bb0 0x7f07a00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a49ea640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f07a00c5fc0 0x7f07a00c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:60028/0 (socket says 10.20.193.18:60028) 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f07a49ea640 1 -- 10.20.193.18:0/3261498587 learned_addr learned my addr 10.20.193.18:0/3261498587 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f079f7fe640 10 monclient(hunting): get_auth_request con 0x7f07a0152bb0 auth_method 0 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f079f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f079f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.168+0000 7f079f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.169+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:31.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.169+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.169+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 1 -- 10.20.193.18:0/3261498587 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92418305 0 0) 0x7f07a006e5e0 con 0x7f07a00c5fc0 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 10 monclient(hunting): my global_id is 5101 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079effd640 1 -- 10.20.193.18:0/3261498587 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0784003620 con 0x7f07a00c5fc0 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 10 monclient(hunting): handle_auth_done global_id 5154 payload 274 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 1 -- 10.20.193.18:0/3261498587 >> v1:10.20.193.32:6789/0 conn(0x7f07a00c5fc0 legacy=0x7f07a00c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 1 -- 10.20.193.18:0/3261498587 >> v1:10.20.193.18:6789/0 conn(0x7f07a01527d0 legacy=0x7f07a006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:31.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:31.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:31.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.170+0000 7f079f7fe640 1 -- 10.20.193.18:0/3261498587 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a006e8c0 con 0x7f07a0152bb0 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.172+0000 7f079f7fe640 1 --2- 10.20.193.18:0/3261498587 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a0152bb0 0x7f07a00c5a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07900098a0 tx=0x7f079002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60b25edb3a436a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 1 -- 10.20.193.18:0/3261498587 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079003c070 con 0x7f07a0152bb0 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: dump: 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.181 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: _finish_auth 0 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 1 -- 10.20.193.18:0/3261498587 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079002fb70 con 0x7f07a0152bb0 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 5 monclient: authenticate success, global_id 5154 2026-02-24T18:40:31.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 1 -- 10.20.193.18:0/3261498587 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790037440 con 0x7f07a0152bb0 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 4 set_mon_vals no callback set 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f079effd640 10 monclient: dump: 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:31.183 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 10 monclient: shutdown 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 1 -- 10.20.193.18:0/3261498587 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a0152bb0 msgr2=0x7f07a00c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.174+0000 7f07a6474640 1 --2- 10.20.193.18:0/3261498587 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a0152bb0 0x7f07a00c5a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07900098a0 tx=0x7f079002ee60 comp rx=0 tx=0).stop 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 -- 10.20.193.18:0/3261498587 shutdown_connections 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 --2- 10.20.193.18:0/3261498587 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a0152bb0 0x7f07a00c5a80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 -- 10.20.193.18:0/3261498587 >> 10.20.193.18:0/3261498587 conn(0x7f07a00cd780 msgr2=0x7f07a00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:31.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 -- 10.20.193.18:0/3261498587 shutdown_connections 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 -- 10.20.193.18:0/3261498587 wait complete. 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 10 monclient: build_initial_monmap 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 10 monclient: monmap: 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:31.185 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 librados: starting msgr at 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 librados: starting objecter 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 Processor -- start 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 -- start start 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 librados: setting wanted keys 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 1 librados: calling monclient init 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.175+0000 7f07a6474640 10 monclient: init 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f07a00c5fc0 0x7f07a01f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-b con 0x7f07a00c5fc0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:31.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-a con 0x7f07a01527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f07a0152bb0 0x7f07a00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): picked mon.noname-c con 0x7f07a0152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f07a006ee80 con 0x7f07a01527d0 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00c6470 con 0x7f07a00c5fc0 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00c9570 con 0x7f07a0152bb0 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a6474640 10 monclient(hunting): authenticate will time out at 1774.857130s 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:31.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f07a00c5fc0 0x7f07a01f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:46434/0 (socket says 10.20.193.18:46434) 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 1 -- 10.20.193.18:0/1291747381 learned_addr learned my addr 10.20.193.18:0/1291747381 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): get_auth_request con 0x7f07a01527d0 auth_method 0 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a49ea640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f07a0152bb0 0x7f07a00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 10 monclient(hunting): get_auth_request con 0x7f07a00c5fc0 auth_method 0 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:31.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a49ea640 10 monclient(hunting): get_auth_request con 0x7f07a0152bb0 auth_method 0 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a49ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a49ea640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f07a49ea640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.176+0000 7f079ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.177+0000 7f079ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.177+0000 7f079ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.177+0000 7f07a49ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.177+0000 7f07a49ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.177+0000 7f07a49ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 10 monclient(hunting): handle_auth_done global_id 5160 payload 824 2026-02-24T18:40:31.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 -- 10.20.193.18:0/1291747381 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f07a0152bb0 msgr2=0x7f07a00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f07a0152bb0 0x7f07a00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 -- 10.20.193.18:0/1291747381 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f07a00c5fc0 msgr2=0x7f07a01f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f07a00c5fc0 0x7f07a01f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.178+0000 7f079f7fe640 1 -- 10.20.193.18:0/1291747381 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a01f4e10 con 0x7f07a01527d0 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.179+0000 7f079f7fe640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0790009130 tx=0x7f07900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.179+0000 7f079cff9640 10 client.?.objecter ms_handle_connect 0x7f07a01527d0 2026-02-24T18:40:31.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.179+0000 7f079cff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079003c040 con 0x7f07a01527d0 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: dump: 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:31.192 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0790042630 con 0x7f07a01527d0 2026-02-24T18:40:31.193 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 5 monclient: authenticate success, global_id 5160 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 10 monclient: _renew_subs 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a01f50f0 con 0x7f07a01527d0 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790042a20 con 0x7f07a01527d0 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079e7fc640 4 set_mon_vals no callback set 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f079cff9640 10 monclient: dump: 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.194 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 10 client.5160.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 10 monclient: _renew_subs 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01f56b0 con 0x7f07a01527d0 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.180+0000 7f07a6474640 1 librados: init done 2026-02-24T18:40:31.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.181+0000 7f07a6474640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:31.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.181+0000 7f07a6474640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:31.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.181+0000 7f07a6474640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.181+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a00c5a80 con 0x7f07a01527d0 2026-02-24T18:40:31.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.285+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f07900413b0 con 0x7f07a01527d0 2026-02-24T18:40:31.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.286+0000 7f079cff9640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 0x7f077007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:31.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.286+0000 7f079ffff640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 0x7f077007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:31.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.286+0000 7f079ffff640 10 monclient: get_auth_request con 0x7f0770078e50 auth_method 0 2026-02-24T18:40:31.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.287+0000 7f079cff9640 10 client.5160.objecter ms_handle_connect 0x7f0770078e50 2026-02-24T18:40:31.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.287+0000 7f079ffff640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 0x7f077007b310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f078c002770 tx=0x7f078c007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:31.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.293+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f079002fab0 con 0x7f07a01527d0 2026-02-24T18:40:31.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.293+0000 7f079cff9640 10 client.5160.objecter ms_dispatch2 0x7f07a0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:31.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.293+0000 7f079cff9640 3 client.5160.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:31.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.293+0000 7f079cff9640 3 client.5160.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:31.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.294+0000 7f079cff9640 20 client.5160.objecter dump_active .. 0 homeless 2026-02-24T18:40:31.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.294+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0790086fc0 con 0x7f07a01527d0 2026-02-24T18:40:31.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.294+0000 7f079cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:31.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.294+0000 7f079cff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:31.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.394+0000 7f07a6474640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:31.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.394+0000 7f07a6474640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:31.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.394+0000 7f07a6474640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:31.397 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.394+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f07a01519a0 con 0x7f07a01527d0 2026-02-24T18:40:31.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.397+0000 7f079cff9640 1 -- 10.20.193.18:0/1291747381 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f079008b2c0 con 0x7f07a01527d0 2026-02-24T18:40:31.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.397+0000 7f079cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:31.400 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.397+0000 7f079cff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:31.400 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:31.401 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:31.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.398+0000 7f07a6474640 10 librados: watch_flush enter 2026-02-24T18:40:31.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.398+0000 7f07a6474640 10 librados: watch_flush exit 2026-02-24T18:40:31.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.398+0000 7f07a6474640 20 client.5160.objecter shutdown clearing up homeless session... 2026-02-24T18:40:31.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.398+0000 7f07a6474640 10 client.5160.objecter successfully canceled tick 2026-02-24T18:40:31.402 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 msgr2=0x7f077007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 0x7f077007b310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f078c002770 tx=0x7f078c007ef0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 10 monclient: shutdown 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 msgr2=0x7f07a00cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0790009130 tx=0x7f07900047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 shutdown_connections 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0770078e50 0x7f077007b310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f07a0152bb0 0x7f07a00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f07a01527d0 0x7f07a00cc380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 --2- 10.20.193.18:0/1291747381 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f07a00c5fc0 0x7f07a01f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:31.403 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 >> 10.20.193.18:0/1291747381 conn(0x7f07a00cd780 msgr2=0x7f07a00c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:31.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 shutdown_connections 2026-02-24T18:40:31.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 -- 10.20.193.18:0/1291747381 wait complete. 2026-02-24T18:40:31.404 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:31.399+0000 7f07a6474640 1 librados: shutdown 2026-02-24T18:40:34.412 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.478+0000 7f05be31b640 10 monclient: get_monmap_and_config 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.478+0000 7f05be31b640 10 monclient: build_initial_monmap 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.478+0000 7f05be31b640 10 monclient: monmap: 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:34.480592+0000 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:34.480592+0000 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 Processor -- start 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 -- start start 2026-02-24T18:40:34.481 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient: init 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient: _add_conns ranks=[4,5,1,2,3,0] 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f05b81527d0 0x7f05b806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-b con 0x7f05b81527d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f05b8152bb0 0x7f05b80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-c con 0x7f05b8152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f05b80c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b806e300 con 0x7f05b80c5fc0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f05b806f1f0 con 0x7f05b81527d0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f05b80c6470 con 0x7f05b8152bb0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.479+0000 7f05be31b640 10 monclient(hunting): authenticate will time out at 1778.161724s 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f05b81527d0 0x7f05b806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f05b8152bb0 0x7f05b80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f05b8152bb0 0x7f05b80c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44358/0 (socket says 10.20.193.18:44358) 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 1 -- 10.20.193.18:0/2012864562 learned_addr learned my addr 10.20.193.18:0/2012864562 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): get_auth_request con 0x7f05b8152bb0 auth_method 0 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:34.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): get_auth_request con 0x7f05b81527d0 auth_method 0 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.480+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:34.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 10 monclient(hunting): handle_auth_done global_id 5090 payload 274 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 1 -- 10.20.193.18:0/2012864562 >> v2:10.20.193.32:3300/0 conn(0x7f05b81527d0 msgr2=0x7f05b806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 1 --2- 10.20.193.18:0/2012864562 >> v2:10.20.193.32:3300/0 conn(0x7f05b81527d0 0x7f05b806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 1 -- 10.20.193.18:0/2012864562 >> v1:10.20.193.32:6789/0 conn(0x7f05b80c5fc0 legacy=0x7f05b80c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 1 monclient: found mon.noname-c 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.481+0000 7f05b77fe640 1 -- 10.20.193.18:0/2012864562 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b806e5e0 con 0x7f05b8152bb0 2026-02-24T18:40:34.484 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.482+0000 7f05b77fe640 1 --2- 10.20.193.18:0/2012864562 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b8152bb0 0x7f05b80c5a80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05b80750e0 tx=0x7f05a802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9728907c73abe015 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 1 -- 10.20.193.18:0/2012864562 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a803c070 con 0x7f05b8152bb0 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: dump: 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:34.486 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 1 -- 10.20.193.18:0/2012864562 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a802fb40 con 0x7f05b8152bb0 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 1 -- 10.20.193.18:0/2012864562 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8037440 con 0x7f05b8152bb0 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 5 monclient: authenticate success, global_id 5090 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 4 set_mon_vals no callback set 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05b6ffd640 10 monclient: dump: 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 10 monclient: shutdown 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 -- 10.20.193.18:0/2012864562 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b8152bb0 msgr2=0x7f05b80c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:34.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 --2- 10.20.193.18:0/2012864562 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b8152bb0 0x7f05b80c5a80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05b80750e0 tx=0x7f05a802eed0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 -- 10.20.193.18:0/2012864562 shutdown_connections 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 --2- 10.20.193.18:0/2012864562 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b8152bb0 0x7f05b80c5a80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 --2- 10.20.193.18:0/2012864562 >> v2:10.20.193.32:3300/0 conn(0x7f05b81527d0 0x7f05b806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 -- 10.20.193.18:0/2012864562 >> 10.20.193.18:0/2012864562 conn(0x7f05b80cd780 msgr2=0x7f05b80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 -- 10.20.193.18:0/2012864562 shutdown_connections 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.484+0000 7f05be31b640 1 -- 10.20.193.18:0/2012864562 wait complete. 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient: build_initial_monmap 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient: monmap: 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 librados: starting msgr at 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 librados: starting objecter 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 Processor -- start 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 -- start start 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 librados: setting wanted keys 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 librados: calling monclient init 2026-02-24T18:40:34.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient: init 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-c con 0x7f05b80c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f05b81527d0 0x7f05b81f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-b con 0x7f05b81527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f05b8152bb0 0x7f05b81f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): picked mon.noname-a con 0x7f05b8152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f05b806ee80 con 0x7f05b8152bb0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05b806eba0 con 0x7f05b81527d0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b806dfa0 con 0x7f05b80c5fc0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.485+0000 7f05be31b640 10 monclient(hunting): authenticate will time out at 1778.167872s 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f05b81527d0 0x7f05b81f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f05b8152bb0 0x7f05b81f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:44370/0 (socket says 10.20.193.18:44370) 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 1 -- 10.20.193.18:0/4148301817 learned_addr learned my addr 10.20.193.18:0/4148301817 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): get_auth_request con 0x7f05b80c5fc0 auth_method 0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): get_auth_request con 0x7f05b81527d0 auth_method 0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): get_auth_request con 0x7f05b8152bb0 auth_method 0 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:34.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.486+0000 7f05bc891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 10 monclient(hunting): handle_auth_done global_id 5093 payload 824 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 -- 10.20.193.18:0/4148301817 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f05b81527d0 msgr2=0x7f05b81f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f05b81527d0 0x7f05b81f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 -- 10.20.193.18:0/4148301817 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f05b8152bb0 msgr2=0x7f05b81f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f05b8152bb0 0x7f05b81f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 monclient: found mon.noname-c 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.487+0000 7f05b7fff640 1 -- 10.20.193.18:0/4148301817 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b81f5280 con 0x7f05b80c5fc0 2026-02-24T18:40:34.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.488+0000 7f05b7fff640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f05a400ec10 tx=0x7f05a400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:34.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.488+0000 7f05b4ff9640 10 client.?.objecter ms_handle_connect 0x7f05b80c5fc0 2026-02-24T18:40:34.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.488+0000 7f05b4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4017070 con 0x7f05b80c5fc0 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: dump: 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 5 monclient: authenticate success, global_id 5093 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 monclient: _renew_subs 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b81f5560 con 0x7f05b80c5fc0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 client.5093.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 monclient: _renew_subs 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81f1870 con 0x7f05b80c5fc0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 1 librados: init done 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a400cc40 con 0x7f05b80c5fc0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4021060 con 0x7f05b80c5fc0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b4ff9640 10 monclient: dump: 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05b67fc640 4 set_mon_vals no callback set 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.490+0000 7f05be31b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.491+0000 7f05be31b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.491+0000 7f05be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.491+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b806da60 con 0x7f05b80c5fc0 2026-02-24T18:40:34.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.501+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f05a4013470 con 0x7f05b80c5fc0 2026-02-24T18:40:34.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.502+0000 7f05b4ff9640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 0x7f058807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:34.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.502+0000 7f05b77fe640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 0x7f058807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:34.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.502+0000 7f05b77fe640 10 monclient: get_auth_request con 0x7f0588078e00 auth_method 0 2026-02-24T18:40:34.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.502+0000 7f05b4ff9640 10 client.5093.objecter ms_handle_connect 0x7f0588078e00 2026-02-24T18:40:34.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.502+0000 7f05b77fe640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 0x7f058807b2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05b8158940 tx=0x7f05a8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:34.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.503+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f05a405e840 con 0x7f05b80c5fc0 2026-02-24T18:40:34.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.503+0000 7f05b4ff9640 10 client.5093.objecter ms_dispatch2 0x7f05b8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:34.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.503+0000 7f05b4ff9640 3 client.5093.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:34.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.503+0000 7f05b4ff9640 3 client.5093.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:34.506 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.503+0000 7f05b4ff9640 20 client.5093.objecter dump_active .. 0 homeless 2026-02-24T18:40:34.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.604+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f05a4062800 con 0x7f05b80c5fc0 2026-02-24T18:40:34.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.604+0000 7f05b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:34.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.604+0000 7f05b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:34.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.703+0000 7f05be31b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:34.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.703+0000 7f05be31b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:34.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.703+0000 7f05be31b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:34.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.703+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f05b80c84e0 con 0x7f05b80c5fc0 2026-02-24T18:40:34.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.706+0000 7f05b4ff9640 1 -- 10.20.193.18:0/4148301817 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f05a4066b00 con 0x7f05b80c5fc0 2026-02-24T18:40:34.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.706+0000 7f05b4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:34.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.706+0000 7f05b4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:34.709 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:34.709 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 10 librados: watch_flush enter 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 10 librados: watch_flush exit 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 20 client.5093.objecter shutdown clearing up homeless session... 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 10 client.5093.objecter successfully canceled tick 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 msgr2=0x7f058807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 0x7f058807b2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05b8158940 tx=0x7f05a8033000 comp rx=0 tx=0).stop 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 10 monclient: shutdown 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 msgr2=0x7f05b81494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:34.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f05a400ec10 tx=0x7f05a400c520 comp rx=0 tx=0).stop 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 shutdown_connections 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f0588078e00 0x7f058807b2c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f05b81527d0 0x7f05b81f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f05b8152bb0 0x7f05b81f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 --2- 10.20.193.18:0/4148301817 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f05b80c5fc0 0x7f05b81494f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 >> 10.20.193.18:0/4148301817 conn(0x7f05b80cd780 msgr2=0x7f05b8150f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 shutdown_connections 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 -- 10.20.193.18:0/4148301817 wait complete. 2026-02-24T18:40:34.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:34.707+0000 7f05be31b640 1 librados: shutdown 2026-02-24T18:40:37.720 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:37.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.789+0000 7f972e274640 10 monclient: get_monmap_and_config 2026-02-24T18:40:37.791 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.789+0000 7f972e274640 10 monclient: build_initial_monmap 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.790+0000 7f972e274640 10 monclient: monmap: 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:37.790842+0000 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:37.790842+0000 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:37.792 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:37.793 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:37.793 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.790+0000 7f972e274640 1 Processor -- start 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.790+0000 7f972e274640 1 -- start start 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.790+0000 7f972e274640 10 monclient: init 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient: _add_conns ranks=[1,3,0,5,4,2] 2026-02-24T18:40:37.795 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f972806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f97280c99a0 0x7f9728070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): picked mon.noname-a con 0x7f97280c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f972815a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97280c7e90 con 0x7f972815a5e0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97280c8170 con 0x7f972806ff20 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f9728159a40 con 0x7f97280c99a0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f972e274640 10 monclient(hunting): authenticate will time out at 1781.472080s 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9726ffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f97280c99a0 0x7f9728070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:37.796 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9727fff640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f972815a5e0 0x7f972814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:35470/0 (socket says 10.20.193.18:35470) 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9727fff640 1 -- 10.20.193.18:0/155508097 learned_addr learned my addr 10.20.193.18:0/155508097 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9726ffd640 10 monclient(hunting): get_auth_request con 0x7f97280c99a0 auth_method 0 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9726ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9726ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.791+0000 7f9726ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.792+0000 7f9726ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.792+0000 7f9726ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.792+0000 7f9726ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 10 monclient(hunting): handle_auth_done global_id 5166 payload 274 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 1 -- 10.20.193.18:0/155508097 >> v1:10.20.193.32:6789/0 conn(0x7f972806ff20 legacy=0x7f9728159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 1 -- 10.20.193.18:0/155508097 >> v1:10.20.193.18:6789/0 conn(0x7f972815a5e0 legacy=0x7f972814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:37.797 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 1 monclient: found mon.noname-a 2026-02-24T18:40:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.793+0000 7f9726ffd640 1 -- 10.20.193.18:0/155508097 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97280c8450 con 0x7f97280c99a0 2026-02-24T18:40:37.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.795+0000 7f9726ffd640 1 --2- 10.20.193.18:0/155508097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f9728070300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f97180098d0 tx=0x7f971802ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92be37a541782868 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 1 -- 10.20.193.18:0/155508097 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971803c070 con 0x7f97280c99a0 2026-02-24T18:40:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:37.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: dump: 2026-02-24T18:40:37.802 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:37.803 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 1 -- 10.20.193.18:0/155508097 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971802fb00 con 0x7f97280c99a0 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 1 -- 10.20.193.18:0/155508097 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718037440 con 0x7f97280c99a0 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f972e274640 5 monclient: authenticate success, global_id 5166 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f972e274640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f972e274640 4 set_mon_vals no callback set 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.797+0000 7f97267fc640 10 monclient: dump: 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:37.804 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 10 monclient: shutdown 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- 10.20.193.18:0/155508097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 msgr2=0x7f9728070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 --2- 10.20.193.18:0/155508097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f9728070300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f97180098d0 tx=0x7f971802ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:37.805 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- 10.20.193.18:0/155508097 shutdown_connections 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 --2- 10.20.193.18:0/155508097 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f9728070300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- 10.20.193.18:0/155508097 >> 10.20.193.18:0/155508097 conn(0x7f97280cd780 msgr2=0x7f97280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- 10.20.193.18:0/155508097 shutdown_connections 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- 10.20.193.18:0/155508097 wait complete. 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 10 monclient: build_initial_monmap 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 10 monclient: monmap: 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:37.806 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 librados: starting msgr at 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 librados: starting objecter 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 Processor -- start 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 -- start start 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 librados: setting wanted keys 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 1 librados: calling monclient init 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.798+0000 7f972e274640 10 monclient: init 2026-02-24T18:40:37.807 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): picked mon.noname-b con 0x7f972806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f97280cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): picked mon.noname-a con 0x7f97280c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f972815a5e0 0x7f97280cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): picked mon.noname-c con 0x7f972815a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9728150520 con 0x7f97280c99a0 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97280c8a10 con 0x7f972806ff20 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:37.808 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97281506a0 con 0x7f972815a5e0 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f972e274640 10 monclient(hunting): authenticate will time out at 1781.480403s 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f97277fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f97277fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:49284/0 (socket says 10.20.193.18:49284) 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f9727fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f972815a5e0 0x7f97280cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f97277fe640 1 -- 10.20.193.18:0/486213577 learned_addr learned my addr 10.20.193.18:0/486213577 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.799+0000 7f97277fe640 10 monclient(hunting): get_auth_request con 0x7f972806ff20 auth_method 0 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:37.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): get_auth_request con 0x7f972815a5e0 auth_method 0 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f97277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:37.810 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.800+0000 7f9727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:37.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 10 monclient(hunting): handle_auth_done global_id 5113 payload 824 2026-02-24T18:40:37.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 -- 10.20.193.18:0/486213577 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f972815a5e0 msgr2=0x7f97280cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f972815a5e0 0x7f97280cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 -- 10.20.193.18:0/486213577 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 msgr2=0x7f97280cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f97280cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 monclient: found mon.noname-b 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.801+0000 7f97277fe640 1 -- 10.20.193.18:0/486213577 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972806f320 con 0x7f972806ff20 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.802+0000 7f97277fe640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f971400b570 tx=0x7f971400ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.802+0000 7f9703fff640 10 client.?.objecter ms_handle_connect 0x7f972806ff20 2026-02-24T18:40:37.813 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.802+0000 7f9703fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714013020 con 0x7f972806ff20 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: dump: 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:37.814 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: _finish_auth 0 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97140044a0 con 0x7f972806ff20 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140029c0 con 0x7f972806ff20 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9725ffb640 4 set_mon_vals no callback set 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f9703fff640 10 monclient: dump: 2026-02-24T18:40:37.815 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 5 monclient: authenticate success, global_id 5113 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 10 monclient: _renew_subs 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:37.816 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972806df50 con 0x7f972806ff20 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 10 client.5113.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 10 monclient: _renew_subs 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972806e510 con 0x7f972806ff20 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.804+0000 7f972e274640 1 librados: init done 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.805+0000 7f972e274640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.805+0000 7f972e274640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.805+0000 7f972e274640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:37.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.805+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9728070300 con 0x7f972806ff20 2026-02-24T18:40:37.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.816+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9714002b60 con 0x7f972806ff20 2026-02-24T18:40:37.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.816+0000 7f9703fff640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 0x7f96f807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:37.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.816+0000 7f9726ffd640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 0x7f96f807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:37.818 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.816+0000 7f9726ffd640 10 monclient: get_auth_request con 0x7f96f8078e00 auth_method 0 2026-02-24T18:40:37.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.817+0000 7f9703fff640 10 client.5113.objecter ms_handle_connect 0x7f96f8078e00 2026-02-24T18:40:37.819 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.817+0000 7f9726ffd640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 0x7f96f807b2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9718039000 tx=0x7f9718031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:37.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.818+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f971405dfa0 con 0x7f972806ff20 2026-02-24T18:40:37.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.818+0000 7f9703fff640 10 client.5113.objecter ms_dispatch2 0x7f9728000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:37.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.818+0000 7f9703fff640 3 client.5113.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:37.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.818+0000 7f9703fff640 3 client.5113.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:37.820 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.818+0000 7f9703fff640 20 client.5113.objecter dump_active .. 0 homeless 2026-02-24T18:40:37.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.917+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f971401d020 con 0x7f972806ff20 2026-02-24T18:40:37.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.917+0000 7f9703fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:37.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:37.917+0000 7f9703fff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:38.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.017+0000 7f972e274640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:38.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.017+0000 7f972e274640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:38.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.017+0000 7f972e274640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:38.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.017+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9728076f20 con 0x7f972806ff20 2026-02-24T18:40:38.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.019+0000 7f9703fff640 1 -- 10.20.193.18:0/486213577 <== mon.1 v2:10.20.193.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9714062660 con 0x7f972806ff20 2026-02-24T18:40:38.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.019+0000 7f9703fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:38.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.019+0000 7f9703fff640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:38.021 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:38.021 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:38.022 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 10 librados: watch_flush enter 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 10 librados: watch_flush exit 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 20 client.5113.objecter shutdown clearing up homeless session... 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 10 client.5113.objecter successfully canceled tick 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 msgr2=0x7f96f807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 0x7f96f807b2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9718039000 tx=0x7f9718031000 comp rx=0 tx=0).stop 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 10 monclient: shutdown 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 msgr2=0x7f972806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f971400b570 tx=0x7f971400ba40 comp rx=0 tx=0).stop 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 shutdown_connections 2026-02-24T18:40:38.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f96f8078e00 0x7f96f807b2c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f972815a5e0 0x7f97280cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f97280c99a0 0x7f97280cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 --2- 10.20.193.18:0/486213577 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f972806ff20 0x7f972806ede0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 >> 10.20.193.18:0/486213577 conn(0x7f97280cd780 msgr2=0x7f97280cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 shutdown_connections 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 -- 10.20.193.18:0/486213577 wait complete. 2026-02-24T18:40:38.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:38.020+0000 7f972e274640 1 librados: shutdown 2026-02-24T18:40:39.742 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:17.688 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.744 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:17.690 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.752 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:17.697 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.771 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:17.716 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.771 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:17.717 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.773 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:17.682 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.774 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:17.683 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.779 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:17.725 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:39.782 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:17.690 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:41.032 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:40:41.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 10 monclient: get_monmap_and_config 2026-02-24T18:40:41.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 10 monclient: monmap: 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:41.103201+0000 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:41.103201+0000 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:41.105 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:41.106 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 1 Processor -- start 2026-02-24T18:40:41.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 1 -- start start 2026-02-24T18:40:41.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.102+0000 7f34977d6640 10 monclient: init 2026-02-24T18:40:41.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient: _add_conns ranks=[2,3,5,0,4,1] 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f349006ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 0x7f3490070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-a con 0x7f3490155eb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f349015c970 0x7f349015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-c con 0x7f349015c970 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34900725b0 con 0x7f349006ff20 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f3490072890 con 0x7f3490155eb0 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3490072b70 con 0x7f349015c970 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f34977d6640 10 monclient(hunting): authenticate will time out at 1784.784321s 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f3494d4a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 0x7f3490070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f3495d4c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f349015c970 0x7f349015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.103+0000 7f3494d4a640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 0x7f3490070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49214/0 (socket says 10.20.193.18:49214) 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 1 -- 10.20.193.18:0/793506559 learned_addr learned my addr 10.20.193.18:0/793506559 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): get_auth_request con 0x7f3490155eb0 auth_method 0 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.110 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): get_auth_request con 0x7f349015c970 auth_method 0 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.104+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 10 monclient(hunting): handle_auth_done global_id 5099 payload 274 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 1 -- 10.20.193.18:0/793506559 >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 msgr2=0x7f3490070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 1 --2- 10.20.193.18:0/793506559 >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 0x7f3490070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.111 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 1 -- 10.20.193.18:0/793506559 >> v1:10.20.193.110:6789/0 conn(0x7f349006ff20 legacy=0x7f34901557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 1 monclient: found mon.noname-c 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.105+0000 7f3495d4c640 1 -- 10.20.193.18:0/793506559 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3490072e50 con 0x7f349015c970 2026-02-24T18:40:41.112 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.106+0000 7f3495d4c640 1 --2- 10.20.193.18:0/793506559 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f349015d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34900c0e90 tx=0x7f348c02f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f75cc8d9fbb9609 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 1 -- 10.20.193.18:0/793506559 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c03c070 con 0x7f349015c970 2026-02-24T18:40:41.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: dump: 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.119 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 1 -- 10.20.193.18:0/793506559 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c02fe30 con 0x7f349015c970 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 1 -- 10.20.193.18:0/793506559 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c037640 con 0x7f349015c970 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f347ffff640 10 monclient: dump: 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.120 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 5 monclient: authenticate success, global_id 5099 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 4 set_mon_vals no callback set 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 10 monclient: shutdown 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.121 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 -- 10.20.193.18:0/793506559 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 msgr2=0x7f349015d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 --2- 10.20.193.18:0/793506559 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f349015d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34900c0e90 tx=0x7f348c02f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 -- 10.20.193.18:0/793506559 shutdown_connections 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 --2- 10.20.193.18:0/793506559 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f349015d1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 --2- 10.20.193.18:0/793506559 >> v2:10.20.193.18:3300/0 conn(0x7f3490155eb0 0x7f3490070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 -- 10.20.193.18:0/793506559 >> 10.20.193.18:0/793506559 conn(0x7f34900cd780 msgr2=0x7f34900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 -- 10.20.193.18:0/793506559 shutdown_connections 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.108+0000 7f34977d6640 1 -- 10.20.193.18:0/793506559 wait complete. 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 10 monclient: monmap: 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:41.122 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 librados: starting msgr at 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 librados: starting objecter 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 Processor -- start 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 -- start start 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 librados: setting wanted keys 2026-02-24T18:40:41.123 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 1 librados: calling monclient init 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.109+0000 7f34977d6640 10 monclient: init 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f349006ff20 0x7f34901f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-b con 0x7f349006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3490155eb0 0x7f34901f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-a con 0x7f3490155eb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): picked mon.noname-c con 0x7f349015c970 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f34900737d0 con 0x7f3490155eb0 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3490073650 con 0x7f349006ff20 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3490073950 con 0x7f349015c970 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f34977d6640 10 monclient(hunting): authenticate will time out at 1784.791088s 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3490155eb0 0x7f34901f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f349554b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f349006ff20 0x7f34901f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37662/0 (socket says 10.20.193.18:37662) 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 1 -- 10.20.193.18:0/2908117974 learned_addr learned my addr 10.20.193.18:0/2908117974 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): get_auth_request con 0x7f349015c970 auth_method 0 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.125 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): get_auth_request con 0x7f3490155eb0 auth_method 0 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3495d4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f349554b640 10 monclient(hunting): get_auth_request con 0x7f349006ff20 auth_method 0 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f349554b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f349554b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f349554b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.110+0000 7f3494d4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.111+0000 7f349554b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.111+0000 7f349554b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.111+0000 7f349554b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 10 monclient(hunting): handle_auth_done global_id 5102 payload 824 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 -- 10.20.193.18:0/2908117974 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f349006ff20 msgr2=0x7f34901f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f349006ff20 0x7f34901f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 -- 10.20.193.18:0/2908117974 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3490155eb0 msgr2=0x7f34901f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3490155eb0 0x7f34901f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 monclient: found mon.noname-c 2026-02-24T18:40:41.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.112+0000 7f3495d4c640 1 -- 10.20.193.18:0/2908117974 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34901f4ce0 con 0x7f349015c970 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.113+0000 7f3495d4c640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f348c02f6d0 tx=0x7f348c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.113+0000 7f347dffb640 10 client.?.objecter ms_handle_connect 0x7f349015c970 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.113+0000 7f347dffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c046070 con 0x7f349015c970 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 10 monclient: dump: 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.128 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f347dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 5 monclient: authenticate success, global_id 5102 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 10 monclient: _renew_subs 2026-02-24T18:40:41.129 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34901e7310 con 0x7f349015c970 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 10 client.5102.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 10 monclient: _renew_subs 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34901e78d0 con 0x7f349015c970 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.114+0000 7f34977d6640 1 librados: init done 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c041460 con 0x7f349015c970 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c050060 con 0x7f349015c970 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347f7fe640 4 set_mon_vals no callback set 2026-02-24T18:40:41.130 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f347dffb640 10 monclient: dump: 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f34977d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f34977d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.131 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f34977d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.132 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.115+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3490070300 con 0x7f349015c970 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.126+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f348c041810 con 0x7f349015c970 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.126+0000 7f347dffb640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 0x7f346007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.127+0000 7f349554b640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 0x7f346007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.127+0000 7f349554b640 10 monclient: get_auth_request con 0x7f3460078e00 auth_method 0 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.127+0000 7f349554b640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 0x7f346007b2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3480004660 tx=0x7f3480009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.127+0000 7f347dffb640 10 client.5102.objecter ms_handle_connect 0x7f3460078e00 2026-02-24T18:40:41.133 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.128+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f348c083d00 con 0x7f349015c970 2026-02-24T18:40:41.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.128+0000 7f347dffb640 10 client.5102.objecter ms_dispatch2 0x7f3490000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:41.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.128+0000 7f347dffb640 3 client.5102.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:41.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.128+0000 7f347dffb640 3 client.5102.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:41.134 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.128+0000 7f347dffb640 20 client.5102.objecter dump_active .. 0 homeless 2026-02-24T18:40:41.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.228+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f348c087cc0 con 0x7f349015c970 2026-02-24T18:40:41.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.228+0000 7f347dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.228+0000 7f347dffb640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:41.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.328+0000 7f34977d6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-24T18:40:41.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.328+0000 7f34977d6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:41.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.328+0000 7f34977d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.328+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f34901596f0 con 0x7f349015c970 2026-02-24T18:40:41.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.330+0000 7f347dffb640 1 -- 10.20.193.18:0/2908117974 <== mon.2 v2:10.20.193.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f348c08bfc0 con 0x7f349015c970 2026-02-24T18:40:41.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.330+0000 7f347dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-24T18:40:41.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.330+0000 7f347dffb640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:41.332 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:41.333 INFO:teuthology.orchestra.run.trial018.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"10 OSD(s) experiencing slow operations in BlueStore","count":10},"muted":false}},"mutes":[]} 2026-02-24T18:40:41.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 10 librados: watch_flush enter 2026-02-24T18:40:41.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 10 librados: watch_flush exit 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 20 client.5102.objecter shutdown clearing up homeless session... 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 10 client.5102.objecter successfully canceled tick 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 msgr2=0x7f346007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 0x7f346007b2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3480004660 tx=0x7f3480009110 comp rx=0 tx=0).stop 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 10 monclient: shutdown 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 msgr2=0x7f34901e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f348c02f6d0 tx=0x7f348c004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 shutdown_connections 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3460078e00 0x7f346007b2c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f349015c970 0x7f34901e83b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3490155eb0 0x7f34901f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 --2- 10.20.193.18:0/2908117974 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f349006ff20 0x7f34901f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 >> 10.20.193.18:0/2908117974 conn(0x7f34900cd780 msgr2=0x7f3490155570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 shutdown_connections 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.331+0000 7f34977d6640 1 -- 10.20.193.18:0/2908117974 wait complete. 2026-02-24T18:40:41.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.332+0000 7f34977d6640 1 librados: shutdown 2026-02-24T18:40:41.342 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T18:40:41.342 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:40:41.411 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 10 monclient: get_monmap_and_config 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 10 monclient: monmap: 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:41.411210+0000 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:41.411210+0000 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.412 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:41.413 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 1 Processor -- start 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 1 -- start start 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.410+0000 7f15849ce640 10 monclient: init 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient: _add_conns ranks=[2,5,1,0,4,3] 2026-02-24T18:40:41.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f157c06ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f157c0c99a0 0x7f157c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-c con 0x7f157c0c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f157c15a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f157c0c7e90 con 0x7f157c15a5e0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f157c0c8170 con 0x7f157c06ff20 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f157c159a40 con 0x7f157c0c99a0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f15849ce640 10 monclient(hunting): authenticate will time out at 1785.092347s 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f1581f42640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f157c0c99a0 0x7f157c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f1582f44640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 0x7f157c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34674/0 (socket says 10.20.193.18:34674) 2026-02-24T18:40:41.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f1582f44640 1 -- 10.20.193.18:0/3854968116 learned_addr learned my addr 10.20.193.18:0/3854968116 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.411+0000 7f1581f42640 10 monclient(hunting): get_auth_request con 0x7f157c0c99a0 auth_method 0 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1582f44640 1 -- 10.20.193.18:0/3854968116 >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 legacy=0x7f157c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1582f44640 1 -- 10.20.193.18:0/3854968116 >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 legacy=0x7f157c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1582f44640 1 --1- 10.20.193.18:0/3854968116 >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 0x7f157c14fe10 :-1 s=OPENED pgs=226 cs=1 l=1).handle_message read tag failed 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1582f44640 1 --1- 10.20.193.18:0/3854968116 >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 0x7f157c14fe10 :-1 s=OPENED pgs=226 cs=1 l=1).fault on lossy channel, failing 2026-02-24T18:40:41.418 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.412+0000 7f1581741640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.32:6789/0 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 10 monclient(hunting): handle_auth_done global_id 5105 payload 274 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 1 -- 10.20.193.18:0/3854968116 >> v1:10.20.193.110:6789/0 conn(0x7f157c06ff20 legacy=0x7f157c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 1 -- 10.20.193.18:0/3854968116 >> v1:10.20.193.32:6789/0 conn(0x7f157c15a5e0 legacy=0x7f157c14fe10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 1 monclient: found mon.noname-c 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.413+0000 7f1581f42640 1 -- 10.20.193.18:0/3854968116 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c0c8450 con 0x7f157c0c99a0 2026-02-24T18:40:41.419 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.415+0000 7f1581f42640 1 --2- 10.20.193.18:0/3854968116 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c070300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f157c0709c0 tx=0x7f156802eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd85d52a44e8c56 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 1 -- 10.20.193.18:0/3854968116 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156803c070 con 0x7f157c0c99a0 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: dump: 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.426 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 1 -- 10.20.193.18:0/3854968116 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156802fa20 con 0x7f157c0c99a0 2026-02-24T18:40:41.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 1 -- 10.20.193.18:0/3854968116 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156802fdf0 con 0x7f157c0c99a0 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f15849ce640 5 monclient: authenticate success, global_id 5105 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f15849ce640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f15849ce640 4 set_mon_vals no callback set 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.416+0000 7f1581741640 10 monclient: dump: 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.428 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 10 monclient: shutdown 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 -- 10.20.193.18:0/3854968116 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 msgr2=0x7f157c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 --2- 10.20.193.18:0/3854968116 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c070300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f157c0709c0 tx=0x7f156802eda0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 -- 10.20.193.18:0/3854968116 shutdown_connections 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 --2- 10.20.193.18:0/3854968116 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c070300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 -- 10.20.193.18:0/3854968116 >> 10.20.193.18:0/3854968116 conn(0x7f157c0cd780 msgr2=0x7f157c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 -- 10.20.193.18:0/3854968116 shutdown_connections 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 1 -- 10.20.193.18:0/3854968116 wait complete. 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.417+0000 7f15849ce640 10 monclient: monmap: 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:41.430 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 librados: starting msgr at 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 librados: starting objecter 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 Processor -- start 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 -- start start 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 librados: setting wanted keys 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 librados: calling monclient init 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient: init 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f157c06ff20 0x7f157c1f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-b con 0x7f157c06ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:41.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-c con 0x7f157c0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f157c15a5e0 0x7f157c1ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): picked mon.noname-a con 0x7f157c15a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f157c150520 con 0x7f157c15a5e0 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f157c0c8a10 con 0x7f157c06ff20 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1506a0 con 0x7f157c0c99a0 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.418+0000 7f15849ce640 10 monclient(hunting): authenticate will time out at 1785.099691s 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f157c15a5e0 0x7f157c1ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f157c06ff20 0x7f157c1f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37682/0 (socket says 10.20.193.18:37682) 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 1 -- 10.20.193.18:0/541327103 learned_addr learned my addr 10.20.193.18:0/541327103 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): get_auth_request con 0x7f157c0c99a0 auth_method 0 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): get_auth_request con 0x7f157c15a5e0 auth_method 0 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): get_auth_request con 0x7f157c06ff20 auth_method 0 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1581f42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582743640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.419+0000 7f1582f44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 10 monclient(hunting): handle_auth_done global_id 5108 payload 824 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 -- 10.20.193.18:0/541327103 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f157c06ff20 msgr2=0x7f157c1f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f157c06ff20 0x7f157c1f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 -- 10.20.193.18:0/541327103 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f157c15a5e0 msgr2=0x7f157c1ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f157c15a5e0 0x7f157c1ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 monclient: found mon.noname-c 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.420+0000 7f1581f42640 1 -- 10.20.193.18:0/541327103 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c1f2560 con 0x7f157c0c99a0 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.422+0000 7f1581f42640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1568009950 tx=0x7f15680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.422+0000 7f1566ffd640 10 client.?.objecter ms_handle_connect 0x7f157c0c99a0 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.422+0000 7f1566ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568046020 con 0x7f157c0c99a0 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: dump: 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.436 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 5 monclient: authenticate success, global_id 5108 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 monclient: _renew_subs 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c1f35b0 con 0x7f157c0c99a0 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 client.5108.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 monclient: _renew_subs 2026-02-24T18:40:41.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1f3b70 con 0x7f157c0c99a0 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 1 librados: init done 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156802fa20 con 0x7f157c0c99a0 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156803c040 con 0x7f157c0c99a0 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1566ffd640 10 monclient: dump: 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.438 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f1580f40640 4 set_mon_vals no callback set 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.423+0000 7f15849ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.424+0000 7f15849ce640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:41.439 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.424+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c070300 con 0x7f157c0c99a0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.435+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f156803f030 con 0x7f157c0c99a0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.435+0000 7f1566ffd640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 0x7f154c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1582743640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 0x7f154c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1582743640 10 monclient: get_auth_request con 0x7f154c078e00 auth_method 0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1582743640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 0x7f154c07b2c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f157c1f5c30 tx=0x7f15740091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 10 client.5108.objecter ms_handle_connect 0x7f154c078e00 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f15680bc740 con 0x7f157c0c99a0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 10 client.5108.objecter ms_dispatch2 0x7f157c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 3 client.5108.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:41.441 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 3 client.5108.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:41.442 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.436+0000 7f1566ffd640 20 client.5108.objecter dump_active .. 0 homeless 2026-02-24T18:40:41.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.535+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1568050030 con 0x7f157c0c99a0 2026-02-24T18:40:41.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.535+0000 7f1566ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.537 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.535+0000 7f1566ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:41.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.616+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f157c14d5c0 con 0x7f154c078e00 2026-02-24T18:40:41.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.617+0000 7f1566ffd640 1 -- 10.20.193.18:0/541327103 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44600 (secure 0 0 0) 0x7f157c14d5c0 con 0x7f154c078e00 2026-02-24T18:40:41.620 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 10 librados: watch_flush enter 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 10 librados: watch_flush exit 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 20 client.5108.objecter shutdown clearing up homeless session... 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 10 client.5108.objecter successfully canceled tick 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 msgr2=0x7f154c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 0x7f154c07b2c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f157c1f5c30 tx=0x7f15740091c0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 10 monclient: shutdown 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 msgr2=0x7f157c149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1568009950 tx=0x7f15680047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 shutdown_connections 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f154c078e00 0x7f154c07b2c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f157c15a5e0 0x7f157c1ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f157c0c99a0 0x7f157c149ec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 --2- 10.20.193.18:0/541327103 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f157c06ff20 0x7f157c1f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 >> 10.20.193.18:0/541327103 conn(0x7f157c0cd780 msgr2=0x7f157c0c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 shutdown_connections 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 -- 10.20.193.18:0/541327103 wait complete. 2026-02-24T18:40:41.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.618+0000 7f15849ce640 1 librados: shutdown 2026-02-24T18:40:41.629 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2026-02-24T18:40:39.856628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5720,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5857280,"data_stored":2838502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001913"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:43.017956+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:37:43.017956+0000","last_peered":"2026-02-24T18:37:43.017956+0000","last_clean":"2026-02-24T18:37:43.017956+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:37:43.017956+0000","last_undegraded":"2026-02-24T18:37:43.017956+0000","last_fullsized":"2026-02-24T18:37:43.017956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:41.029076+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:37:41.029076+0000","last_peered":"2026-02-24T18:37:41.029076+0000","last_clean":"2026-02-24T18:37:41.029076+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:37:41.029076+0000","last_undegraded":"2026-02-24T18:37:41.029076+0000","last_fullsized":"2026-02-24T18:37:41.029076+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.978048+0000","last_change":"2026-02-24T18:35:32.450105+0000","last_active":"2026-02-24T18:37:37.978048+0000","last_peered":"2026-02-24T18:37:37.978048+0000","last_clean":"2026-02-24T18:37:37.978048+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:37:37.978048+0000","last_undegraded":"2026-02-24T18:37:37.978048+0000","last_fullsized":"2026-02-24T18:37:37.978048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:08:13.272990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:34.995215+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:37:34.995215+0000","last_peered":"2026-02-24T18:37:34.995215+0000","last_clean":"2026-02-24T18:37:34.995215+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:37:34.995215+0000","last_undegraded":"2026-02-24T18:37:34.995215+0000","last_fullsized":"2026-02-24T18:37:34.995215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.965070+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:37:37.965070+0000","last_peered":"2026-02-24T18:37:37.965070+0000","last_clean":"2026-02-24T18:37:37.965070+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:37:37.965070+0000","last_undegraded":"2026-02-24T18:37:37.965070+0000","last_fullsized":"2026-02-24T18:37:37.965070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.980866+0000","last_change":"2026-02-24T18:35:33.340094+0000","last_active":"2026-02-24T18:37:37.980866+0000","last_peered":"2026-02-24T18:37:37.980866+0000","last_clean":"2026-02-24T18:37:37.980866+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:37:37.980866+0000","last_undegraded":"2026-02-24T18:37:37.980866+0000","last_fullsized":"2026-02-24T18:37:37.980866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:14.613111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:32.962781+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:37:32.962781+0000","last_peered":"2026-02-24T18:37:32.962781+0000","last_clean":"2026-02-24T18:37:32.962781+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:37:32.962781+0000","last_undegraded":"2026-02-24T18:37:32.962781+0000","last_fullsized":"2026-02-24T18:37:32.962781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:35.992355+0000","last_change":"2026-02-24T18:35:32.447249+0000","last_active":"2026-02-24T18:37:35.992355+0000","last_peered":"2026-02-24T18:37:35.992355+0000","last_clean":"2026-02-24T18:37:35.992355+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:37:35.992355+0000","last_undegraded":"2026-02-24T18:37:35.992355+0000","last_fullsized":"2026-02-24T18:37:35.992355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:27:55.602090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"28'32","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:27.975403+0000","last_change":"2026-02-24T18:35:28.845690+0000","last_active":"2026-02-24T18:37:27.975403+0000","last_peered":"2026-02-24T18:37:27.975403+0000","last_clean":"2026-02-24T18:37:27.975403+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:37:27.975403+0000","last_undegraded":"2026-02-24T18:37:27.975403+0000","last_fullsized":"2026-02-24T18:37:27.975403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_clean_scrub_stamp":"2026-02-24T18:35:26.937667+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:13.898984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280577,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:40:41.631 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:40:41.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 10 monclient: get_monmap_and_config 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 10 monclient: monmap: 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:41.701025+0000 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:41.701025+0000 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:41.702 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:41.703 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:41.703 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:41.703 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:41.703 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 1 Processor -- start 2026-02-24T18:40:41.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 1 -- start start 2026-02-24T18:40:41.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.700+0000 7f892ed03640 10 monclient: init 2026-02-24T18:40:41.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.706 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient: _add_conns ranks=[5,3,4,2,1,0] 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f892806ff20 0x7f8928070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-c con 0x7f892806ff20 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 0x7f8928159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-a con 0x7f89280c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f892815a410 0x7f892814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-b con 0x7f892815a410 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f8928159a40 con 0x7f89280c99a0 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f892815a860 con 0x7f892815a410 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f89280c7e90 con 0x7f892806ff20 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ed03640 10 monclient(hunting): authenticate will time out at 1785.382165s 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f892815a410 0x7f892814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ca78640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f892806ff20 0x7f8928070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 0x7f8928159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 0x7f8928159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49228/0 (socket says 10.20.193.18:49228) 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f892815a410 0x7f892814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52072/0 (socket says 10.20.193.18:52072) 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 1 -- 10.20.193.18:0/4056191289 learned_addr learned my addr 10.20.193.18:0/4056191289 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 10 monclient(hunting): get_auth_request con 0x7f89280c99a0 auth_method 0 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f891ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 10 monclient(hunting): get_auth_request con 0x7f892815a410 auth_method 0 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892d279640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ca78640 10 monclient(hunting): get_auth_request con 0x7f892806ff20 auth_method 0 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ca78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ca78640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.701+0000 7f892ca78640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.702+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 10 monclient(hunting): handle_auth_done global_id 5122 payload 274 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 -- 10.20.193.18:0/4056191289 >> v2:10.20.193.110:3300/0 conn(0x7f892806ff20 msgr2=0x7f8928070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 --2- 10.20.193.18:0/4056191289 >> v2:10.20.193.110:3300/0 conn(0x7f892806ff20 0x7f8928070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 -- 10.20.193.18:0/4056191289 >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 msgr2=0x7f8928159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 --2- 10.20.193.18:0/4056191289 >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 0x7f8928159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 monclient: found mon.noname-b 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:41.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.703+0000 7f892d279640 1 -- 10.20.193.18:0/4056191289 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89280c8170 con 0x7f892815a410 2026-02-24T18:40:41.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.704+0000 7f892d279640 1 --2- 10.20.193.18:0/4056191289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892815a410 0x7f892814fe10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f89280709c0 tx=0x7f892402f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501d4361a1f1b775 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 1 -- 10.20.193.18:0/4056191289 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892403c070 con 0x7f892815a410 2026-02-24T18:40:41.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.717 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: dump: 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.718 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 1 -- 10.20.193.18:0/4056191289 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892402fe40 con 0x7f892815a410 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 5 monclient: authenticate success, global_id 5122 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 1 -- 10.20.193.18:0/4056191289 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924037660 con 0x7f892815a410 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 4 set_mon_vals no callback set 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f891f7fe640 10 monclient: dump: 2026-02-24T18:40:41.719 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 10 monclient: shutdown 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.720 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 -- 10.20.193.18:0/4056191289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892815a410 msgr2=0x7f892814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 --2- 10.20.193.18:0/4056191289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892815a410 0x7f892814fe10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f89280709c0 tx=0x7f892402f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 -- 10.20.193.18:0/4056191289 shutdown_connections 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 --2- 10.20.193.18:0/4056191289 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892815a410 0x7f892814fe10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 --2- 10.20.193.18:0/4056191289 >> v2:10.20.193.18:3300/0 conn(0x7f89280c99a0 0x7f8928159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 --2- 10.20.193.18:0/4056191289 >> v2:10.20.193.110:3300/0 conn(0x7f892806ff20 0x7f8928070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 -- 10.20.193.18:0/4056191289 >> 10.20.193.18:0/4056191289 conn(0x7f89280cd780 msgr2=0x7f89280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 -- 10.20.193.18:0/4056191289 shutdown_connections 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.706+0000 7f892ed03640 1 -- 10.20.193.18:0/4056191289 wait complete. 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient: monmap: 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.721 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 librados: starting msgr at 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 librados: starting objecter 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 Processor -- start 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 -- start start 2026-02-24T18:40:41.722 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 librados: setting wanted keys 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 librados: calling monclient init 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient: init 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 0x7f8928156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-b con 0x7f892806ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f89280c99a0 0x7f89281569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-c con 0x7f89280c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 0x7f892814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): picked mon.noname-a con 0x7f892815a410 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f89280c8a10 con 0x7f892815a410 2026-02-24T18:40:41.723 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89280c8730 con 0x7f892806ff20 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8928150350 con 0x7f89280c99a0 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.707+0000 7f892ed03640 10 monclient(hunting): authenticate will time out at 1785.388721s 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 0x7f892814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 0x7f8928156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 0x7f892814cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49238/0 (socket says 10.20.193.18:49238) 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 1 -- 10.20.193.18:0/592491177 learned_addr learned my addr 10.20.193.18:0/592491177 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): get_auth_request con 0x7f892815a410 auth_method 0 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.724 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f89280c99a0 0x7f89281569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): get_auth_request con 0x7f892806ff20 auth_method 0 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892d279640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): get_auth_request con 0x7f89280c99a0 auth_method 0 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f892ca78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.708+0000 7f891ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 10 monclient(hunting): handle_auth_done global_id 5125 payload 824 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 -- 10.20.193.18:0/592491177 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f89280c99a0 msgr2=0x7f89281569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f89280c99a0 0x7f89281569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 -- 10.20.193.18:0/592491177 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 msgr2=0x7f892814cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 0x7f892814cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 monclient: found mon.noname-b 2026-02-24T18:40:41.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.710+0000 7f892ca78640 1 -- 10.20.193.18:0/592491177 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89281f55e0 con 0x7f892806ff20 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.711+0000 7f892ca78640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 0x7f8928156490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f891800e9b0 tx=0x7f891800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.711+0000 7f891d7fa640 10 client.?.objecter ms_handle_connect 0x7f892806ff20 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.711+0000 7f891d7fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918017070 con 0x7f892806ff20 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: dump: 2026-02-24T18:40:41.727 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 5 monclient: authenticate success, global_id 5125 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 monclient: _renew_subs 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89281f58c0 con 0x7f892806ff20 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 client.5125.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 monclient: _renew_subs 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89281f5e80 con 0x7f892806ff20 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 1 librados: init done 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891800ca00 con 0x7f892806ff20 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918021060 con 0x7f892806ff20 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891d7fa640 10 monclient: dump: 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.730 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f891effd640 4 set_mon_vals no callback set 2026-02-24T18:40:41.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.712+0000 7f892ed03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.713+0000 7f892ed03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.713+0000 7f892ed03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:41.731 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.713+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89281506e0 con 0x7f892806ff20 2026-02-24T18:40:41.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.724+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8918026480 con 0x7f892806ff20 2026-02-24T18:40:41.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.724+0000 7f891d7fa640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 0x7f88f807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.724+0000 7f891ffff640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 0x7f88f807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.724+0000 7f891ffff640 10 monclient: get_auth_request con 0x7f88f8078e00 auth_method 0 2026-02-24T18:40:41.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891ffff640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 0x7f88f807b2c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89281f77f0 tx=0x7f8910009340 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891d7fa640 10 client.5125.objecter ms_handle_connect 0x7f88f8078e00 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f891800cdb0 con 0x7f892806ff20 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891d7fa640 10 client.5125.objecter ms_dispatch2 0x7f8928000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891d7fa640 3 client.5125.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.725+0000 7f891d7fa640 3 client.5125.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:41.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.726+0000 7f891d7fa640 20 client.5125.objecter dump_active .. 0 homeless 2026-02-24T18:40:41.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.825+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8918062580 con 0x7f892806ff20 2026-02-24T18:40:41.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.825+0000 7f891d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:41.827 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.825+0000 7f891d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:41.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.907+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8928159e70 con 0x7f88f8078e00 2026-02-24T18:40:41.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f891d7fa640 1 -- 10.20.193.18:0/592491177 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44600 (secure 0 0 0) 0x7f8928159e70 con 0x7f88f8078e00 2026-02-24T18:40:41.911 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f892ed03640 10 librados: watch_flush enter 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f892ed03640 10 librados: watch_flush exit 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f892ed03640 20 client.5125.objecter shutdown clearing up homeless session... 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f892ed03640 10 client.5125.objecter successfully canceled tick 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.909+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 msgr2=0x7f88f807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 0x7f88f807b2c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89281f77f0 tx=0x7f8910009340 comp rx=0 tx=0).stop 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 10 monclient: shutdown 2026-02-24T18:40:41.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 msgr2=0x7f8928156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 0x7f8928156490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f891800e9b0 tx=0x7f891800ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 shutdown_connections 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f88f8078e00 0x7f88f807b2c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f892815a410 0x7f892814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f89280c99a0 0x7f89281569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 --2- 10.20.193.18:0/592491177 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f892806ff20 0x7f8928156490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 >> 10.20.193.18:0/592491177 conn(0x7f89280cd780 msgr2=0x7f89281584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 shutdown_connections 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 -- 10.20.193.18:0/592491177 wait complete. 2026-02-24T18:40:41.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.910+0000 7f892ed03640 1 librados: shutdown 2026-02-24T18:40:41.920 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-24T18:40:41.856876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5720,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5857280,"data_stored":2838502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001823"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:43.017956+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:37:43.017956+0000","last_peered":"2026-02-24T18:37:43.017956+0000","last_clean":"2026-02-24T18:37:43.017956+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:37:43.017956+0000","last_undegraded":"2026-02-24T18:37:43.017956+0000","last_fullsized":"2026-02-24T18:37:43.017956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:41.029076+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:37:41.029076+0000","last_peered":"2026-02-24T18:37:41.029076+0000","last_clean":"2026-02-24T18:37:41.029076+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:37:41.029076+0000","last_undegraded":"2026-02-24T18:37:41.029076+0000","last_fullsized":"2026-02-24T18:37:41.029076+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.978048+0000","last_change":"2026-02-24T18:35:32.450105+0000","last_active":"2026-02-24T18:37:37.978048+0000","last_peered":"2026-02-24T18:37:37.978048+0000","last_clean":"2026-02-24T18:37:37.978048+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:37:37.978048+0000","last_undegraded":"2026-02-24T18:37:37.978048+0000","last_fullsized":"2026-02-24T18:37:37.978048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:08:13.272990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:34.995215+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:37:34.995215+0000","last_peered":"2026-02-24T18:37:34.995215+0000","last_clean":"2026-02-24T18:37:34.995215+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:37:34.995215+0000","last_undegraded":"2026-02-24T18:37:34.995215+0000","last_fullsized":"2026-02-24T18:37:34.995215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.965070+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:37:37.965070+0000","last_peered":"2026-02-24T18:37:37.965070+0000","last_clean":"2026-02-24T18:37:37.965070+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:37:37.965070+0000","last_undegraded":"2026-02-24T18:37:37.965070+0000","last_fullsized":"2026-02-24T18:37:37.965070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.980866+0000","last_change":"2026-02-24T18:35:33.340094+0000","last_active":"2026-02-24T18:37:37.980866+0000","last_peered":"2026-02-24T18:37:37.980866+0000","last_clean":"2026-02-24T18:37:37.980866+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:37:37.980866+0000","last_undegraded":"2026-02-24T18:37:37.980866+0000","last_fullsized":"2026-02-24T18:37:37.980866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:14.613111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:32.962781+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:37:32.962781+0000","last_peered":"2026-02-24T18:37:32.962781+0000","last_clean":"2026-02-24T18:37:32.962781+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:37:32.962781+0000","last_undegraded":"2026-02-24T18:37:32.962781+0000","last_fullsized":"2026-02-24T18:37:32.962781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:35.992355+0000","last_change":"2026-02-24T18:35:32.447249+0000","last_active":"2026-02-24T18:37:35.992355+0000","last_peered":"2026-02-24T18:37:35.992355+0000","last_clean":"2026-02-24T18:37:35.992355+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:37:35.992355+0000","last_undegraded":"2026-02-24T18:37:35.992355+0000","last_fullsized":"2026-02-24T18:37:35.992355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:27:55.602090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"28'32","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:27.975403+0000","last_change":"2026-02-24T18:35:28.845690+0000","last_active":"2026-02-24T18:37:27.975403+0000","last_peered":"2026-02-24T18:37:27.975403+0000","last_clean":"2026-02-24T18:37:27.975403+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:37:27.975403+0000","last_undegraded":"2026-02-24T18:37:27.975403+0000","last_fullsized":"2026-02-24T18:37:27.975403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_clean_scrub_stamp":"2026-02-24T18:35:26.937667+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:13.898984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:40:41.922 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T18:40:41.922 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.989+0000 7f7569703640 10 monclient: get_monmap_and_config 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.989+0000 7f7569703640 10 monclient: build_initial_monmap 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.989+0000 7f7569703640 10 monclient: monmap: 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:41.990677+0000 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:41.990677+0000 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:41.991 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 Processor -- start 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 -- start start 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient: init 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient: _add_conns ranks=[1,0,3,2,5,4] 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f756406ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f75640c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f756415a5e0 0x7f7564070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): picked mon.noname-a con 0x7f756415a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75640c7e90 con 0x7f75640c99a0 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75640c8170 con 0x7f756406ff20 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7564159a40 con 0x7f756415a5e0 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:41.992 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.990+0000 7f7569703640 10 monclient(hunting): authenticate will time out at 1785.671837s 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f75637fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f756415a5e0 0x7f7564070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f7562ffd640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f756406ff20 0x7f7564159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34676/0 (socket says 10.20.193.18:34676) 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f7562ffd640 1 -- 10.20.193.18:0/2761723271 learned_addr learned my addr 10.20.193.18:0/2761723271 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f75637fe640 10 monclient(hunting): get_auth_request con 0x7f756415a5e0 auth_method 0 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f75637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f75637fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.991+0000 7f75637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 1 -- 10.20.193.18:0/2761723271 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69426402 0 0) 0x7f75640c8170 con 0x7f756406ff20 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:41.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 10 monclient(hunting): my global_id is 5128 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.992+0000 7f7561ffb640 1 -- 10.20.193.18:0/2761723271 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7550003620 con 0x7f756406ff20 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 10 monclient(hunting): handle_auth_done global_id 5187 payload 274 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 1 -- 10.20.193.18:0/2761723271 >> v1:10.20.193.32:6789/0 conn(0x7f756406ff20 legacy=0x7f7564159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 1 -- 10.20.193.18:0/2761723271 >> v1:10.20.193.18:6789/0 conn(0x7f75640c99a0 legacy=0x7f756414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:41.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.993+0000 7f75637fe640 1 -- 10.20.193.18:0/2761723271 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75640c8450 con 0x7f756415a5e0 2026-02-24T18:40:41.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.995+0000 7f75637fe640 1 --2- 10.20.193.18:0/2761723271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f756415a5e0 0x7f7564070300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7554009dc0 tx=0x7f755402f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e27473cd0dbc00 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 1 -- 10.20.193.18:0/2761723271 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755403c070 con 0x7f756415a5e0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: dump: 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 1 -- 10.20.193.18:0/2761723271 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7554037440 con 0x7f756415a5e0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 1 -- 10.20.193.18:0/2761723271 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554037810 con 0x7f756415a5e0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 5 monclient: authenticate success, global_id 5187 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 4 set_mon_vals no callback set 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7561ffb640 10 monclient: dump: 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:41.999 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 10 monclient: shutdown 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 -- 10.20.193.18:0/2761723271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f756415a5e0 msgr2=0x7f7564070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 --2- 10.20.193.18:0/2761723271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f756415a5e0 0x7f7564070300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7554009dc0 tx=0x7f755402f280 comp rx=0 tx=0).stop 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 -- 10.20.193.18:0/2761723271 shutdown_connections 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 --2- 10.20.193.18:0/2761723271 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f756415a5e0 0x7f7564070300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 -- 10.20.193.18:0/2761723271 >> 10.20.193.18:0/2761723271 conn(0x7f75640cd780 msgr2=0x7f75640cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.997+0000 7f7569703640 1 -- 10.20.193.18:0/2761723271 shutdown_connections 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 -- 10.20.193.18:0/2761723271 wait complete. 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 10 monclient: build_initial_monmap 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 10 monclient: monmap: 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 librados: starting msgr at 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 librados: starting objecter 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 Processor -- start 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 -- start start 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 librados: setting wanted keys 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 1 librados: calling monclient init 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.998+0000 7f7569703640 10 monclient: init 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:42.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f756406ff20 0x7f75640cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): picked mon.noname-c con 0x7f756406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): picked mon.noname-a con 0x7f75640c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f756415a5e0 0x7f75641f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): picked mon.noname-b con 0x7f756415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7564150520 con 0x7f75640c99a0 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75640c8a10 con 0x7f756415a5e0 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75641506a0 con 0x7f756406ff20 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7569703640 10 monclient(hunting): authenticate will time out at 1785.680099s 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49256/0 (socket says 10.20.193.18:49256) 2026-02-24T18:40:42.002 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 1 -- 10.20.193.18:0/4145099145 learned_addr learned my addr 10.20.193.18:0/4145099145 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): get_auth_request con 0x7f75640c99a0 auth_method 0 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7562ffd640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f756406ff20 0x7f75640cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75637fe640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f756415a5e0 0x7f75641f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7562ffd640 10 monclient(hunting): get_auth_request con 0x7f756406ff20 auth_method 0 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7562ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7562ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f7562ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75637fe640 10 monclient(hunting): get_auth_request con 0x7f756415a5e0 auth_method 0 2026-02-24T18:40:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75637fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:41.999+0000 7f75637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f7562ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f7562ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f7562ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.000+0000 7f75637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 10 monclient(hunting): handle_auth_done global_id 5193 payload 824 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 -- 10.20.193.18:0/4145099145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f756406ff20 msgr2=0x7f75640cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f756406ff20 0x7f75640cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 -- 10.20.193.18:0/4145099145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f756415a5e0 msgr2=0x7f75641f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f756415a5e0 0x7f75641f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 monclient: found mon.noname-a 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.001+0000 7f75627fc640 1 -- 10.20.193.18:0/4145099145 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641f4d30 con 0x7f75640c99a0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.002+0000 7f75627fc640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c730 tx=0x7f754c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.002+0000 7f75437fe640 10 client.?.objecter ms_handle_connect 0x7f75640c99a0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.002+0000 7f75437fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0052d0 con 0x7f75640c99a0 2026-02-24T18:40:42.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: dump: 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754c005470 con 0x7f75640c99a0 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0038d0 con 0x7f75640c99a0 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75437fe640 10 monclient: dump: 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.010 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f75617fa640 4 set_mon_vals no callback set 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 5 monclient: authenticate success, global_id 5193 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _renew_subs 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641f5010 con 0x7f75640c99a0 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 client.5193.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:42.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _renew_subs 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641f55d0 con 0x7f75640c99a0 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 1 librados: init done 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.004+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7564070300 con 0x7f75640c99a0 2026-02-24T18:40:42.017 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.016+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f754c003a70 con 0x7f75640c99a0 2026-02-24T18:40:42.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.016+0000 7f75437fe640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 0x7f753007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.016+0000 7f7562ffd640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 0x7f753007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.016+0000 7f7562ffd640 10 monclient: get_auth_request con 0x7f7530078e30 auth_method 0 2026-02-24T18:40:42.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.017+0000 7f7562ffd640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 0x7f753007b2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7558007b60 tx=0x7f7558007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:42.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.017+0000 7f75437fe640 10 client.5193.objecter ms_handle_connect 0x7f7530078e30 2026-02-24T18:40:42.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.018+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f754c05e030 con 0x7f75640c99a0 2026-02-24T18:40:42.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.018+0000 7f75437fe640 10 client.5193.objecter ms_dispatch2 0x7f7564000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:42.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.018+0000 7f75437fe640 3 client.5193.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:42.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.018+0000 7f75437fe640 3 client.5193.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:42.020 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.018+0000 7f75437fe640 20 client.5193.objecter dump_active .. 0 homeless 2026-02-24T18:40:42.118 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.117+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f754c061ff0 con 0x7f75640c99a0 2026-02-24T18:40:42.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.117+0000 7f75437fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.119 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.117+0000 7f75437fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:42.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.198+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75641f4ae0 con 0x7f7530078e30 2026-02-24T18:40:42.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.199+0000 7f75437fe640 1 -- 10.20.193.18:0/4145099145 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44600 (secure 0 0 0) 0x7f75641f4ae0 con 0x7f7530078e30 2026-02-24T18:40:42.202 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:42.203 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:40:42.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 10 librados: watch_flush enter 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 10 librados: watch_flush exit 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 20 client.5193.objecter shutdown clearing up homeless session... 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 10 client.5193.objecter successfully canceled tick 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 msgr2=0x7f753007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 0x7f753007b2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7558007b60 tx=0x7f7558007ab0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 10 monclient: shutdown 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 msgr2=0x7f75641f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c730 tx=0x7f754c00cc00 comp rx=0 tx=0).stop 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 shutdown_connections 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7530078e30 0x7f753007b2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f756415a5e0 0x7f75641f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f75640c99a0 0x7f75641f17e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 --2- 10.20.193.18:0/4145099145 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f756406ff20 0x7f75640cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 >> 10.20.193.18:0/4145099145 conn(0x7f75640cd780 msgr2=0x7f756414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 shutdown_connections 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 -- 10.20.193.18:0/4145099145 wait complete. 2026-02-24T18:40:42.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.200+0000 7f7569703640 1 librados: shutdown 2026-02-24T18:40:42.211 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-24T18:40:41.856876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5720,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5857280,"data_stored":2838502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001823"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:43.017956+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:37:43.017956+0000","last_peered":"2026-02-24T18:37:43.017956+0000","last_clean":"2026-02-24T18:37:43.017956+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:37:43.017956+0000","last_undegraded":"2026-02-24T18:37:43.017956+0000","last_fullsized":"2026-02-24T18:37:43.017956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:41.029076+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:37:41.029076+0000","last_peered":"2026-02-24T18:37:41.029076+0000","last_clean":"2026-02-24T18:37:41.029076+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:37:41.029076+0000","last_undegraded":"2026-02-24T18:37:41.029076+0000","last_fullsized":"2026-02-24T18:37:41.029076+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.978048+0000","last_change":"2026-02-24T18:35:32.450105+0000","last_active":"2026-02-24T18:37:37.978048+0000","last_peered":"2026-02-24T18:37:37.978048+0000","last_clean":"2026-02-24T18:37:37.978048+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:37:37.978048+0000","last_undegraded":"2026-02-24T18:37:37.978048+0000","last_fullsized":"2026-02-24T18:37:37.978048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:08:13.272990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:34.995215+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:37:34.995215+0000","last_peered":"2026-02-24T18:37:34.995215+0000","last_clean":"2026-02-24T18:37:34.995215+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:37:34.995215+0000","last_undegraded":"2026-02-24T18:37:34.995215+0000","last_fullsized":"2026-02-24T18:37:34.995215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.965070+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:37:37.965070+0000","last_peered":"2026-02-24T18:37:37.965070+0000","last_clean":"2026-02-24T18:37:37.965070+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:37:37.965070+0000","last_undegraded":"2026-02-24T18:37:37.965070+0000","last_fullsized":"2026-02-24T18:37:37.965070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.980866+0000","last_change":"2026-02-24T18:35:33.340094+0000","last_active":"2026-02-24T18:37:37.980866+0000","last_peered":"2026-02-24T18:37:37.980866+0000","last_clean":"2026-02-24T18:37:37.980866+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:37:37.980866+0000","last_undegraded":"2026-02-24T18:37:37.980866+0000","last_fullsized":"2026-02-24T18:37:37.980866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:14.613111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:32.962781+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:37:32.962781+0000","last_peered":"2026-02-24T18:37:32.962781+0000","last_clean":"2026-02-24T18:37:32.962781+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:37:32.962781+0000","last_undegraded":"2026-02-24T18:37:32.962781+0000","last_fullsized":"2026-02-24T18:37:32.962781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:35.992355+0000","last_change":"2026-02-24T18:35:32.447249+0000","last_active":"2026-02-24T18:37:35.992355+0000","last_peered":"2026-02-24T18:37:35.992355+0000","last_clean":"2026-02-24T18:37:35.992355+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:37:35.992355+0000","last_undegraded":"2026-02-24T18:37:35.992355+0000","last_fullsized":"2026-02-24T18:37:35.992355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:27:55.602090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"28'32","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:27.975403+0000","last_change":"2026-02-24T18:35:28.845690+0000","last_active":"2026-02-24T18:37:27.975403+0000","last_peered":"2026-02-24T18:37:27.975403+0000","last_clean":"2026-02-24T18:37:27.975403+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:37:27.975403+0000","last_undegraded":"2026-02-24T18:37:27.975403+0000","last_fullsized":"2026-02-24T18:37:27.975403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:26.937667+0000","last_clean_scrub_stamp":"2026-02-24T18:35:26.937667+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:13.898984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:40:42.213 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:40:42.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.280+0000 7fb7893e4640 10 monclient: get_monmap_and_config 2026-02-24T18:40:42.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.280+0000 7fb7893e4640 10 monclient: build_initial_monmap 2026-02-24T18:40:42.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.280+0000 7fb7893e4640 10 monclient: monmap: 2026-02-24T18:40:42.282 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:42.281572+0000 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:42.281572+0000 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:42.283 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:42.284 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 Processor -- start 2026-02-24T18:40:42.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 -- start start 2026-02-24T18:40:42.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient: init 2026-02-24T18:40:42.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:42.287 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient: _add_conns ranks=[1,2,3,5,4,0] 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb78406ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb7840c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb78415a5e0 0x7fb784070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-a con 0x7fb78415a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840c7e90 con 0x7fb78406ff20 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840c8170 con 0x7fb7840c99a0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fb784159a40 con 0x7fb78415a5e0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.281+0000 7fb7893e4640 10 monclient(hunting): authenticate will time out at 1785.962742s 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fb78415a5e0 0x7fb784070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.288 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb782ffd640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fb78406ff20 0x7fb784159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34692/0 (socket says 10.20.193.18:34692) 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb782ffd640 1 -- 10.20.193.18:0/2119355517 learned_addr learned my addr 10.20.193.18:0/2119355517 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): get_auth_request con 0x7fb78415a5e0 auth_method 0 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.282+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716373352 0 0) 0x7fb7840c7e90 con 0x7fb78406ff20 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:42.289 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 10 monclient(hunting): my global_id is 5134 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb770003620 con 0x7fb78406ff20 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 10 monclient(hunting): handle_auth_done global_id 5196 payload 274 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 1 -- 10.20.193.18:0/2119355517 >> v1:10.20.193.110:6789/0 conn(0x7fb7840c99a0 legacy=0x7fb78414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410718979 0 0) 0x7fb7840c8170 con 0x7fb7840c99a0 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 1 -- 10.20.193.18:0/2119355517 >> v1:10.20.193.32:6789/0 conn(0x7fb78406ff20 legacy=0x7fb784159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb7837fe640 1 -- 10.20.193.18:0/2119355517 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840c8450 con 0x7fb78415a5e0 2026-02-24T18:40:42.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.283+0000 7fb781ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.284+0000 7fb7837fe640 1 --2- 10.20.193.18:0/2119355517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb78415a5e0 0x7fb784070300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb774009dc0 tx=0x7fb77402f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17181112fdb201d5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77403c070 con 0x7fb78415a5e0 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 10 monclient: dump: 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.291 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb781ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb7893e4640 5 monclient: authenticate success, global_id 5196 2026-02-24T18:40:42.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.285+0000 7fb7893e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:42.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774037440 con 0x7fb78415a5e0 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 1 -- 10.20.193.18:0/2119355517 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740377f0 con 0x7fb78415a5e0 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb781ffb640 10 monclient: dump: 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.298 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 4 set_mon_vals no callback set 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 10 monclient: shutdown 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 -- 10.20.193.18:0/2119355517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb78415a5e0 msgr2=0x7fb784070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 --2- 10.20.193.18:0/2119355517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb78415a5e0 0x7fb784070300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb774009dc0 tx=0x7fb77402f280 comp rx=0 tx=0).stop 2026-02-24T18:40:42.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 -- 10.20.193.18:0/2119355517 shutdown_connections 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 --2- 10.20.193.18:0/2119355517 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb78415a5e0 0x7fb784070300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 -- 10.20.193.18:0/2119355517 >> 10.20.193.18:0/2119355517 conn(0x7fb7840cd780 msgr2=0x7fb7840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 -- 10.20.193.18:0/2119355517 shutdown_connections 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.286+0000 7fb7893e4640 1 -- 10.20.193.18:0/2119355517 wait complete. 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 10 monclient: build_initial_monmap 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 10 monclient: monmap: 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:42.300 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 librados: starting msgr at 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 librados: starting objecter 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 Processor -- start 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 -- start start 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 librados: setting wanted keys 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 1 librados: calling monclient init 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.287+0000 7fb7893e4640 10 monclient: init 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:42.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb78406ff20 0x7fb7840c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-b con 0x7fb78406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-a con 0x7fb7840c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb78415a5e0 0x7fb7841f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): picked mon.noname-c con 0x7fb78415a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb784150520 con 0x7fb7840c99a0 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840c8a10 con 0x7fb78406ff20 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841506a0 con 0x7fb78415a5e0 2026-02-24T18:40:42.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7893e4640 10 monclient(hunting): authenticate will time out at 1785.969130s 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7837fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb78415a5e0 0x7fb7841f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb782ffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb78406ff20 0x7fb7840c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49278/0 (socket says 10.20.193.18:49278) 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 1 -- 10.20.193.18:0/3677679565 learned_addr learned my addr 10.20.193.18:0/3677679565 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): get_auth_request con 0x7fb7840c99a0 auth_method 0 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb782ffd640 10 monclient(hunting): get_auth_request con 0x7fb78406ff20 auth_method 0 2026-02-24T18:40:42.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb782ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb782ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb782ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7837fe640 10 monclient(hunting): get_auth_request con 0x7fb78415a5e0 auth_method 0 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.288+0000 7fb7827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb782ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb782ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb782ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.289+0000 7fb7837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 10 monclient(hunting): handle_auth_done global_id 5199 payload 824 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 -- 10.20.193.18:0/3677679565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb78415a5e0 msgr2=0x7fb7841f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb78415a5e0 0x7fb7841f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 -- 10.20.193.18:0/3677679565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb78406ff20 msgr2=0x7fb7840c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb78406ff20 0x7fb7840c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 monclient: found mon.noname-a 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.290+0000 7fb7827fc640 1 -- 10.20.193.18:0/3677679565 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840c2780 con 0x7fb7840c99a0 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.291+0000 7fb7827fc640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c730 tx=0x7fb76c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.291+0000 7fb7637fe640 10 client.?.objecter ms_handle_connect 0x7fb7840c99a0 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.291+0000 7fb7637fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c017070 con 0x7fb7840c99a0 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: dump: 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.306 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c005270 con 0x7fb7840c99a0 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c003670 con 0x7fb7840c99a0 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 5 monclient: authenticate success, global_id 5199 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 10 monclient: _renew_subs 2026-02-24T18:40:42.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7817fa640 4 set_mon_vals no callback set 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7840c2a60 con 0x7fb7840c99a0 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7637fe640 10 monclient: dump: 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:42.308 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 10 client.5199.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 10 monclient: _renew_subs 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7840c3020 con 0x7fb7840c99a0 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.292+0000 7fb7893e4640 1 librados: init done 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.293+0000 7fb7893e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.293+0000 7fb7893e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.293+0000 7fb7893e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.293+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb784076e00 con 0x7fb7840c99a0 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.304+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb76c00a5e0 con 0x7fb7840c99a0 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.304+0000 7fb7637fe640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 0x7fb75007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.305+0000 7fb782ffd640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 0x7fb75007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.305+0000 7fb782ffd640 10 monclient: get_auth_request con 0x7fb750078e50 auth_method 0 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.305+0000 7fb782ffd640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 0x7fb75007b310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb778007b60 tx=0x7fb778007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:42.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.305+0000 7fb7637fe640 10 client.5199.objecter ms_handle_connect 0x7fb750078e50 2026-02-24T18:40:42.405 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fb76c05d4d0 con 0x7fb7840c99a0 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 10 client.5199.objecter ms_dispatch2 0x7fb784000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 3 client.5199.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 3 client.5199.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 20 client.5199.objecter dump_active .. 0 homeless 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb784076e00 con 0x7fb7840c99a0 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:42.406 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.403+0000 7fb7637fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:42.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.487+0000 7fb7893e4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:40:42.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.487+0000 7fb7893e4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:40:42.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.487+0000 7fb7893e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:42.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.487+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7840c4af0 con 0x7fb7840c99a0 2026-02-24T18:40:42.507 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.505+0000 7fb7637fe640 1 -- 10.20.193.18:0/3677679565 <== mon.0 v2:10.20.193.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+24664 (secure 0 0 0) 0x7fb76c021020 con 0x7fb7840c99a0 2026-02-24T18:40:42.507 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.505+0000 7fb7637fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-24T18:40:42.507 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.505+0000 7fb7637fe640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:40:42.507 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:42.508 INFO:teuthology.orchestra.run.trial018.stdout:{"epoch":29,"fsid":"8c4cd187-4cf3-4491-88ba-bf63c2f42ea1","created":"2026-02-24T18:33:45.040017+0000","modified":"2026-02-24T18:35:39.217670+0000","last_up_change":"2026-02-24T18:35:26.937667+0000","last_in_change":"2026-02-24T18:33:51.286558+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:35:26.061331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:35:29.058214+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"29f6c438-7191-4921-a473-a7c822283cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6817","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6819","nonce":1076077393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6821","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6823","nonce":1076077393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6829","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6831","nonce":1076077393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6825","nonce":1076077393},{"type":"v1","addr":"10.20.193.18:6827","nonce":1076077393}]},"public_addr":"10.20.193.18:6819/1076077393","cluster_addr":"10.20.193.18:6823/1076077393","heartbeat_back_addr":"10.20.193.18:6831/1076077393","heartbeat_front_addr":"10.20.193.18:6827/1076077393","state":["exists","up"]},{"osd":1,"uuid":"9df55bdb-d545-4f6a-a8c5-816acf7fbe1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6800","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6801","nonce":2795974188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6802","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6803","nonce":2795974188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6806","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6807","nonce":2795974188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6804","nonce":2795974188},{"type":"v1","addr":"10.20.193.32:6805","nonce":2795974188}]},"public_addr":"10.20.193.32:6801/2795974188","cluster_addr":"10.20.193.32:6803/2795974188","heartbeat_back_addr":"10.20.193.32:6807/2795974188","heartbeat_front_addr":"10.20.193.32:6805/2795974188","state":["exists","up"]},{"osd":2,"uuid":"9361a34f-9434-4f10-9037-2c32be178beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6825","nonce":3650974883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6826","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6827","nonce":3650974883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6830","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6831","nonce":3650974883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":3650974883},{"type":"v1","addr":"10.20.193.110:6829","nonce":3650974883}]},"public_addr":"10.20.193.110:6825/3650974883","cluster_addr":"10.20.193.110:6827/3650974883","heartbeat_back_addr":"10.20.193.110:6831/3650974883","heartbeat_front_addr":"10.20.193.110:6829/3650974883","state":["exists","up"]},{"osd":3,"uuid":"733635aa-8ae4-43d7-a53a-f6a66a520fe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6826","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6827","nonce":230320724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6828","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6829","nonce":230320724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6832","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6833","nonce":230320724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6830","nonce":230320724},{"type":"v1","addr":"10.20.193.112:6831","nonce":230320724}]},"public_addr":"10.20.193.112:6827/230320724","cluster_addr":"10.20.193.112:6829/230320724","heartbeat_back_addr":"10.20.193.112:6833/230320724","heartbeat_front_addr":"10.20.193.112:6831/230320724","state":["exists","up"]},{"osd":4,"uuid":"56308d78-75f5-4853-ac45-b958a4dc7cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6800","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6801","nonce":3481454300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6802","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6803","nonce":3481454300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6806","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6807","nonce":3481454300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6804","nonce":3481454300},{"type":"v1","addr":"10.20.193.18:6805","nonce":3481454300}]},"public_addr":"10.20.193.18:6801/3481454300","cluster_addr":"10.20.193.18:6803/3481454300","heartbeat_back_addr":"10.20.193.18:6807/3481454300","heartbeat_front_addr":"10.20.193.18:6805/3481454300","state":["exists","up"]},{"osd":5,"uuid":"9d335fc6-9910-49e7-b3b4-2a66c119eebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6808","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6809","nonce":2128834550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6810","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6811","nonce":2128834550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6814","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6815","nonce":2128834550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6812","nonce":2128834550},{"type":"v1","addr":"10.20.193.32:6813","nonce":2128834550}]},"public_addr":"10.20.193.32:6809/2128834550","cluster_addr":"10.20.193.32:6811/2128834550","heartbeat_back_addr":"10.20.193.32:6815/2128834550","heartbeat_front_addr":"10.20.193.32:6813/2128834550","state":["exists","up"]},{"osd":6,"uuid":"1c4e2c48-941e-4185-9331-6f37534247c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6801","nonce":3466222224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6803","nonce":3466222224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6807","nonce":3466222224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":3466222224},{"type":"v1","addr":"10.20.193.110:6805","nonce":3466222224}]},"public_addr":"10.20.193.110:6801/3466222224","cluster_addr":"10.20.193.110:6803/3466222224","heartbeat_back_addr":"10.20.193.110:6807/3466222224","heartbeat_front_addr":"10.20.193.110:6805/3466222224","state":["exists","up"]},{"osd":7,"uuid":"ec86bc7e-386d-4547-83df-9db486b50617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6810","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6811","nonce":92089351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6814","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6815","nonce":92089351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6821","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6823","nonce":92089351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6817","nonce":92089351},{"type":"v1","addr":"10.20.193.112:6819","nonce":92089351}]},"public_addr":"10.20.193.112:6811/92089351","cluster_addr":"10.20.193.112:6815/92089351","heartbeat_back_addr":"10.20.193.112:6823/92089351","heartbeat_front_addr":"10.20.193.112:6819/92089351","state":["exists","up"]},{"osd":8,"uuid":"2986fffd-4bee-4a9b-847e-b217d2ac3cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6808","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6809","nonce":1977979333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6810","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6811","nonce":1977979333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6814","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6815","nonce":1977979333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6812","nonce":1977979333},{"type":"v1","addr":"10.20.193.18:6813","nonce":1977979333}]},"public_addr":"10.20.193.18:6809/1977979333","cluster_addr":"10.20.193.18:6811/1977979333","heartbeat_back_addr":"10.20.193.18:6815/1977979333","heartbeat_front_addr":"10.20.193.18:6813/1977979333","state":["exists","up"]},{"osd":9,"uuid":"16e6895e-d37b-4a74-be40-45204e7d1077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6816","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6817","nonce":1806399530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6818","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6819","nonce":1806399530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6822","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6823","nonce":1806399530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6820","nonce":1806399530},{"type":"v1","addr":"10.20.193.32:6821","nonce":1806399530}]},"public_addr":"10.20.193.32:6817/1806399530","cluster_addr":"10.20.193.32:6819/1806399530","heartbeat_back_addr":"10.20.193.32:6823/1806399530","heartbeat_front_addr":"10.20.193.32:6821/1806399530","state":["exists","up"]},{"osd":10,"uuid":"8d89aa64-bc98-4a27-8ece-99b034a9095c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6809","nonce":4149932150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6811","nonce":4149932150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6815","nonce":4149932150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":4149932150},{"type":"v1","addr":"10.20.193.110:6813","nonce":4149932150}]},"public_addr":"10.20.193.110:6809/4149932150","cluster_addr":"10.20.193.110:6811/4149932150","heartbeat_back_addr":"10.20.193.110:6815/4149932150","heartbeat_front_addr":"10.20.193.110:6813/4149932150","state":["exists","up"]},{"osd":11,"uuid":"9a649a86-160c-47fc-bcdf-ca545fc0d62a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6812","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6813","nonce":2071089960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6816","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6818","nonce":2071089960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6824","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6825","nonce":2071089960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6820","nonce":2071089960},{"type":"v1","addr":"10.20.193.112:6822","nonce":2071089960}]},"public_addr":"10.20.193.112:6813/2071089960","cluster_addr":"10.20.193.112:6818/2071089960","heartbeat_back_addr":"10.20.193.112:6825/2071089960","heartbeat_front_addr":"10.20.193.112:6822/2071089960","state":["exists","up"]},{"osd":12,"uuid":"53546e48-a93a-4f5f-a114-9ea08210aba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6816","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6818","nonce":392025864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6820","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6822","nonce":392025864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6828","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6830","nonce":392025864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.18:6824","nonce":392025864},{"type":"v1","addr":"10.20.193.18:6826","nonce":392025864}]},"public_addr":"10.20.193.18:6818/392025864","cluster_addr":"10.20.193.18:6822/392025864","heartbeat_back_addr":"10.20.193.18:6830/392025864","heartbeat_front_addr":"10.20.193.18:6826/392025864","state":["exists","up"]},{"osd":13,"uuid":"c6a44a9d-ae2f-4d6f-8fbe-21de7fc51e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6824","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6825","nonce":316721302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6826","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6827","nonce":316721302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6830","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6831","nonce":316721302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.32:6828","nonce":316721302},{"type":"v1","addr":"10.20.193.32:6829","nonce":316721302}]},"public_addr":"10.20.193.32:6825/316721302","cluster_addr":"10.20.193.32:6827/316721302","heartbeat_back_addr":"10.20.193.32:6831/316721302","heartbeat_front_addr":"10.20.193.32:6829/316721302","state":["exists","up"]},{"osd":14,"uuid":"7a952355-2344-492b-aa45-dd403a3f5c2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6817","nonce":3577843273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6818","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6819","nonce":3577843273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6822","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6823","nonce":3577843273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":3577843273},{"type":"v1","addr":"10.20.193.110:6821","nonce":3577843273}]},"public_addr":"10.20.193.110:6817/3577843273","cluster_addr":"10.20.193.110:6819/3577843273","heartbeat_back_addr":"10.20.193.110:6823/3577843273","heartbeat_front_addr":"10.20.193.110:6821/3577843273","state":["exists","up"]},{"osd":15,"uuid":"a565a854-0fc5-46c1-ad7a-350d27bb20d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6802","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6803","nonce":4220209204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6804","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6805","nonce":4220209204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6808","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6809","nonce":4220209204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.112:6806","nonce":4220209204},{"type":"v1","addr":"10.20.193.112:6807","nonce":4220209204}]},"public_addr":"10.20.193.112:6803/4220209204","cluster_addr":"10.20.193.112:6805/4220209204","heartbeat_back_addr":"10.20.193.112:6809/4220209204","heartbeat_front_addr":"10.20.193.112:6807/4220209204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.022230+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.519970+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.248082+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.777136+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.483378+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.749510+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.414866+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.123257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.766447+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.874254+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.504376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.120504+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.013826+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:25.378335+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:23.904134+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:35:24.610284+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:40:42.509 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 10 librados: watch_flush enter 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 10 librados: watch_flush exit 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 20 client.5199.objecter shutdown clearing up homeless session... 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 10 client.5199.objecter successfully canceled tick 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 msgr2=0x7fb75007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 0x7fb75007b310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb778007b60 tx=0x7fb778007ab0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 10 monclient: shutdown 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 msgr2=0x7fb784158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c730 tx=0x7fb76c00cc00 comp rx=0 tx=0).stop 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 shutdown_connections 2026-02-24T18:40:42.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fb750078e50 0x7fb75007b310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fb78415a5e0 0x7fb7841f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fb7840c99a0 0x7fb784158c30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 --2- 10.20.193.18:0/3677679565 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fb78406ff20 0x7fb7840c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 >> 10.20.193.18:0/3677679565 conn(0x7fb7840cd780 msgr2=0x7fb78414e370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 shutdown_connections 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 -- 10.20.193.18:0/3677679565 wait complete. 2026-02-24T18:40:42.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:42.506+0000 7fb7893e4640 1 librados: shutdown 2026-02-24T18:40:43.518 INFO:tasks.ceph:Scrubbing osd.0 2026-02-24T18:40:43.519 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 10 monclient: get_monmap_and_config 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 10 monclient: build_initial_monmap 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 10 monclient: monmap: 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:43.591285+0000 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:43.591285+0000 2026-02-24T18:40:43.592 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:43.593 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 1 Processor -- start 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 1 -- start start 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.590+0000 7fd3dfa09640 10 monclient: init 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient: _add_conns ranks=[4,1,2,3,0,5] 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fd3d81503e0 0x7fd3d8159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-b con 0x7fd3d81503e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd3d815a070 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd3d80704c0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d815a570 con 0x7fd3d815a070 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d815a850 con 0x7fd3d80704c0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d80c7d70 con 0x7fd3d81503e0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dfa09640 10 monclient(hunting): authenticate will time out at 1787.272448s 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.591+0000 7fd3dd77e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fd3d81503e0 0x7fd3d8159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3ddf7f640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fd3d80704c0 0x7fd3d80c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:41740/0 (socket says 10.20.193.18:41740) 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/3203069821 learned_addr learned my addr 10.20.193.18:0/3203069821 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:43.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): get_auth_request con 0x7fd3d81503e0 auth_method 0 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.592+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 1 -- 10.20.193.18:0/3203069821 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56066285 0 0) 0x7fd3d815a850 con 0x7fd3d80704c0 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 10 monclient(hunting): _negotiate 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 10 monclient(hunting): my global_id is 5126 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3c7fff640 1 -- 10.20.193.18:0/3203069821 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3c0003620 con 0x7fd3d80704c0 2026-02-24T18:40:43.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_done global_id 5140 payload 274 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 1 -- 10.20.193.18:0/3203069821 >> v1:10.20.193.110:6789/0 conn(0x7fd3d80704c0 legacy=0x7fd3d80c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 1 -- 10.20.193.18:0/3203069821 >> v1:10.20.193.32:6789/0 conn(0x7fd3d815a070 legacy=0x7fd3d806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 10 monclient: _finish_hunting 0 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 1 monclient: found mon.noname-b 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.593+0000 7fd3dd77e640 1 -- 10.20.193.18:0/3203069821 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d80c7fc0 con 0x7fd3d81503e0 2026-02-24T18:40:43.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.594+0000 7fd3dd77e640 1 --2- 10.20.193.18:0/3203069821 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d81503e0 0x7fd3d8159b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80098a0 tx=0x7fd3c802eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f0a03adb8e4033 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 1 -- 10.20.193.18:0/3203069821 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c803c070 con 0x7fd3d81503e0 2026-02-24T18:40:43.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: dump: 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.605 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: _finish_auth 0 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 1 -- 10.20.193.18:0/3203069821 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c802fb70 con 0x7fd3d81503e0 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 1 -- 10.20.193.18:0/3203069821 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8037440 con 0x7fd3d81503e0 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3c7fff640 10 monclient: dump: 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.606 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 5 monclient: authenticate success, global_id 5140 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 4 set_mon_vals no callback set 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 10 monclient: shutdown 2026-02-24T18:40:43.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 -- 10.20.193.18:0/3203069821 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d81503e0 msgr2=0x7fd3d8159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 --2- 10.20.193.18:0/3203069821 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d81503e0 0x7fd3d8159b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80098a0 tx=0x7fd3c802eeb0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 -- 10.20.193.18:0/3203069821 shutdown_connections 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 --2- 10.20.193.18:0/3203069821 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d81503e0 0x7fd3d8159b30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 -- 10.20.193.18:0/3203069821 >> 10.20.193.18:0/3203069821 conn(0x7fd3d80cd600 msgr2=0x7fd3d80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 -- 10.20.193.18:0/3203069821 shutdown_connections 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.597+0000 7fd3dfa09640 1 -- 10.20.193.18:0/3203069821 wait complete. 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient: build_initial_monmap 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient: monmap: 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:43.608 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 librados: starting msgr at 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 librados: starting objecter 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 Processor -- start 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 -- start start 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 librados: setting wanted keys 2026-02-24T18:40:43.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 librados: calling monclient init 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient: init 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd3d80704c0 0x7fd3d81ec340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-a con 0x7fd3d80704c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3d81503e0 0x7fd3d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-c con 0x7fd3d81503e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 0x7fd3d81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): picked mon.noname-b con 0x7fd3d815a070 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:43.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81507c0 con 0x7fd3d80704c0 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d80c8610 con 0x7fd3d815a070 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8150940 con 0x7fd3d81503e0 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.598+0000 7fd3dfa09640 10 monclient(hunting): authenticate will time out at 1787.279844s 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd3d80704c0 0x7fd3d81ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 0x7fd3d81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3d81503e0 0x7fd3d806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 0x7fd3d81f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52108/0 (socket says 10.20.193.18:52108) 2026-02-24T18:40:43.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/2884196256 learned_addr learned my addr 10.20.193.18:0/2884196256 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): get_auth_request con 0x7fd3d815a070 auth_method 0 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): get_auth_request con 0x7fd3d80704c0 auth_method 0 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): get_auth_request con 0x7fd3d81503e0 auth_method 0 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3ddf7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dd77e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.599+0000 7fd3dcf7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 10 monclient(hunting): handle_auth_done global_id 5143 payload 824 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/2884196256 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3d81503e0 msgr2=0x7fd3d806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd3d81503e0 0x7fd3d806d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd3d80704c0 msgr2=0x7fd3d81ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd3d80704c0 0x7fd3d81ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 10 monclient: _finish_hunting 0 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 monclient: found mon.noname-b 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d806dae0 con 0x7fd3d815a070 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3ddf7f640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 0x7fd3d81f3390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cbb0 tx=0x7fd3d400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3c5ffb640 10 client.?.objecter ms_handle_connect 0x7fd3d815a070 2026-02-24T18:40:43.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.601+0000 7fd3c5ffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:43.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400ad50 con 0x7fd3d815a070 2026-02-24T18:40:43.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:43.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: dump: 2026-02-24T18:40:43.616 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:43.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 5 monclient: authenticate success, global_id 5143 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 10 monclient: _renew_subs 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d4012400 con 0x7fd3d815a070 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d806ddc0 con 0x7fd3d815a070 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4012870 con 0x7fd3d815a070 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c5ffb640 10 monclient: dump: 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.618 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3c77fe640 4 set_mon_vals no callback set 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 10 client.5143.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 10 monclient: _renew_subs 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:43.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d806e380 con 0x7fd3d815a070 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.603+0000 7fd3dfa09640 1 librados: init done 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 10 client.5143.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 15 client.5143.objecter _session_command_op_assign -1 1 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 20 client.5143.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 10 client.5143.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:43.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.604+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3d8157eb0 con 0x7fd3d815a070 2026-02-24T18:40:43.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.615+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd3d4012a10 con 0x7fd3d815a070 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.616+0000 7fd3c5ffb640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd3a8078e50 0x7fd3a807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.616+0000 7fd3dd77e640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd3a8078e50 0x7fd3a807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.616+0000 7fd3dd77e640 10 monclient: get_auth_request con 0x7fd3a8078e50 auth_method 0 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd3d4012d30 con 0x7fd3d815a070 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 10 client.5143.objecter ms_dispatch2 0x7fd3d8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 3 client.5143.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 3 client.5143.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 10 client.5143.objecter checking command 1 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fd3a8081330 0x7fd3a8083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter _get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 15 client.5143.objecter _session_command_op_remove -1 1 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter _get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 15 client.5143.objecter _session_command_op_assign 0 1 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 4 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 10 client.5143.objecter _send_command 1 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3a8083e20 con 0x7fd3a8081330 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 20 client.5143.objecter dump_active .. 0 homeless 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3d405eb50 con 0x7fd3d815a070 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3c5ffb640 10 client.5143.objecter ms_handle_connect 0x7fd3a8078e50 2026-02-24T18:40:43.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3dd77e640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd3a8078e50 0x7fd3a807b310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8039000 tx=0x7fd3c8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3dcf7d640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fd3a8081330 0x7fd3a8083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.617+0000 7fd3dcf7d640 10 monclient: get_auth_request con 0x7fd3a8081330 auth_method 0 2026-02-24T18:40:43.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.619+0000 7fd3dcf7d640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fd3a8081330 0x7fd3a8083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.619+0000 7fd3c5ffb640 10 client.5143.objecter ms_handle_connect 0x7fd3a8081330 2026-02-24T18:40:43.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.628+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== osd.0 v2:10.20.193.18:6817/1076077393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd3a8083e20 con 0x7fd3a8081330 2026-02-24T18:40:43.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.628+0000 7fd3c5ffb640 10 client.5143.objecter ms_dispatch2 0x7fd3d8000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:43.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.628+0000 7fd3c5ffb640 10 client.5143.objecter _finish_command 1 = system:0 2026-02-24T18:40:43.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.628+0000 7fd3c5ffb640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 4 2026-02-24T18:40:43.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.628+0000 7fd3c5ffb640 15 client.5143.objecter _session_command_op_remove 0 1 2026-02-24T18:40:43.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 10 client.5143.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:43.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 15 client.5143.objecter _session_command_op_assign -1 2 2026-02-24T18:40:43.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 20 client.5143.objecter _get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 20 client.5143.objecter _get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 15 client.5143.objecter _session_command_op_remove -1 2 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 20 client.5143.objecter get_session s=0x7fd3a8080c60 osd=0 3 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 15 client.5143.objecter _session_command_op_assign 0 2 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 4 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 10 client.5143.objecter _send_command 2 2026-02-24T18:40:43.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.634+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 --> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd3d8159b30 con 0x7fd3a8081330 2026-02-24T18:40:43.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3c5ffb640 1 -- 10.20.193.18:0/2884196256 <== osd.0 v2:10.20.193.18:6817/1076077393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd3d8159b30 con 0x7fd3a8081330 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3c5ffb640 10 client.5143.objecter ms_dispatch2 0x7fd3d8000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3c5ffb640 10 client.5143.objecter _finish_command 2 = system:0 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3c5ffb640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 4 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3c5ffb640 15 client.5143.objecter _session_command_op_remove 0 2 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:43.650 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:43.651 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 10 librados: watch_flush enter 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 10 librados: watch_flush exit 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 10 client.5143.objecter close_session for osd.0 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fd3a8081330 msgr2=0x7fd3a8083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.18:6817/1076077393,v1:10.20.193.18:6819/1076077393] conn(0x7fd3a8081330 0x7fd3a8083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 20 client.5143.objecter put_session s=0x7fd3a8080c60 osd=0 1 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 20 client.5143.objecter shutdown clearing up homeless session... 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 10 client.5143.objecter successfully canceled tick 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd3a8078e50 msgr2=0x7fd3a807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd3a8078e50 0x7fd3a807b310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8039000 tx=0x7fd3c8033000 comp rx=0 tx=0).stop 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 10 monclient: shutdown 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 msgr2=0x7fd3d81f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 --2- 10.20.193.18:0/2884196256 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd3d815a070 0x7fd3d81f3390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cbb0 tx=0x7fd3d400a430 comp rx=0 tx=0).stop 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3ddf7f640 1 -- 10.20.193.18:0/2884196256 reap_dead start 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 shutdown_connections 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 >> 10.20.193.18:0/2884196256 conn(0x7fd3d80cd600 msgr2=0x7fd3d814f120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:43.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 shutdown_connections 2026-02-24T18:40:43.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 -- 10.20.193.18:0/2884196256 wait complete. 2026-02-24T18:40:43.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.648+0000 7fd3dfa09640 1 librados: shutdown 2026-02-24T18:40:43.658 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-24T18:40:43.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 10 monclient: get_monmap_and_config 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 10 monclient: build_initial_monmap 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 10 monclient: monmap: 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:43.727127+0000 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:43.727127+0000 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.728 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:43.729 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 1 Processor -- start 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 1 -- start start 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.726+0000 7fc7ba4cd640 10 monclient: init 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient: _add_conns ranks=[1,3,2,5,4,0] 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc7b406ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc7b40c99a0 0x7fc7b4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.732 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-a con 0x7fc7b40c99a0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc7b415a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40c7e90 con 0x7fc7b406ff20 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40c8170 con 0x7fc7b415a5e0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b4159a40 con 0x7fc7b40c99a0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7ba4cd640 10 monclient(hunting): authenticate will time out at 1787.408359s 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7b37fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fc7b40c99a0 0x7fc7b4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7b3fff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fc7b406ff20 0x7fc7b4159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34716/0 (socket says 10.20.193.18:34716) 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.727+0000 7fc7b3fff640 1 -- 10.20.193.18:0/905442682 learned_addr learned my addr 10.20.193.18:0/905442682 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): get_auth_request con 0x7fc7b40c99a0 auth_method 0 2026-02-24T18:40:43.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.728+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354562844 0 0) 0x7fc7b40c8170 con 0x7fc7b415a5e0 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): my global_id is 5132 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc798003620 con 0x7fc7b415a5e0 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149189270 0 0) 0x7fc7b40c7e90 con 0x7fc7b406ff20 2026-02-24T18:40:43.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 10 monclient(hunting): my global_id is 5146 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7b40c8170 con 0x7fc7b406ff20 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_done global_id 5205 payload 274 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 1 -- 10.20.193.18:0/905442682 >> v1:10.20.193.110:6789/0 conn(0x7fc7b415a5e0 legacy=0x7fc7b414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 1 -- 10.20.193.18:0/905442682 >> v1:10.20.193.32:6789/0 conn(0x7fc7b406ff20 legacy=0x7fc7b4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:43.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.729+0000 7fc7b37fe640 1 -- 10.20.193.18:0/905442682 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40c8450 con 0x7fc7b40c99a0 2026-02-24T18:40:43.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.730+0000 7fc7b37fe640 1 --2- 10.20.193.18:0/905442682 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4070300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40098d0 tx=0x7fc7a402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1a110c2c43b729 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a403c070 con 0x7fc7b40c99a0 2026-02-24T18:40:43.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:43.742 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: dump: 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a402fba0 con 0x7fc7b40c99a0 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 1 -- 10.20.193.18:0/905442682 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4037440 con 0x7fc7b40c99a0 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7ba4cd640 5 monclient: authenticate success, global_id 5205 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7ba4cd640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7ba4cd640 4 set_mon_vals no callback set 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:43.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.731+0000 7fc7b2ffd640 10 monclient: dump: 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 10 monclient: shutdown 2026-02-24T18:40:43.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/905442682 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b40c99a0 msgr2=0x7fc7b4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/905442682 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4070300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40098d0 tx=0x7fc7a402eed0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/905442682 shutdown_connections 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/905442682 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4070300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/905442682 >> 10.20.193.18:0/905442682 conn(0x7fc7b40cd780 msgr2=0x7fc7b40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/905442682 shutdown_connections 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/905442682 wait complete. 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 10 monclient: build_initial_monmap 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 10 monclient: monmap: 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:43.746 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 librados: starting msgr at 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 librados: starting objecter 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 Processor -- start 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 -- start start 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 librados: setting wanted keys 2026-02-24T18:40:43.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.732+0000 7fc7ba4cd640 1 librados: calling monclient init 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient: init 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc7b406ff20 0x7fc7b41f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-b con 0x7fc7b406ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-c con 0x7fc7b40c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): picked mon.noname-a con 0x7fc7b415a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4150520 con 0x7fc7b415a5e0 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40c8a10 con 0x7fc7b406ff20 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41506a0 con 0x7fc7b40c99a0 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7ba4cd640 10 monclient(hunting): authenticate will time out at 1787.414409s 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49304/0 (socket says 10.20.193.18:49304) 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 1 -- 10.20.193.18:0/2566090600 learned_addr learned my addr 10.20.193.18:0/2566090600 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b3fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc7b406ff20 0x7fc7b41f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 10 monclient(hunting): get_auth_request con 0x7fc7b415a5e0 auth_method 0 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b8a43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.733+0000 7fc7b37fe640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b8a43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b8a43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b8a43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request con 0x7fc7b406ff20 auth_method 0 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): get_auth_request con 0x7fc7b40c99a0 auth_method 0 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:43.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.734+0000 7fc7b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 10 monclient(hunting): handle_auth_done global_id 5208 payload 824 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 -- 10.20.193.18:0/2566090600 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc7b40c99a0 msgr2=0x7fc7b4072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 -- 10.20.193.18:0/2566090600 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc7b406ff20 msgr2=0x7fc7b41f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc7b406ff20 0x7fc7b41f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 10 monclient: _finish_hunting 0 2026-02-24T18:40:43.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 monclient: found mon.noname-a 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.735+0000 7fc7b8a43640 1 -- 10.20.193.18:0/2566090600 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b414d240 con 0x7fc7b415a5e0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.736+0000 7fc7b8a43640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800cbb0 tx=0x7fc7a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.736+0000 7fc7b0ff9640 10 client.?.objecter ms_handle_connect 0x7fc7b415a5e0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.736+0000 7fc7b0ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800ad50 con 0x7fc7b415a5e0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: dump: 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.752 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a8003a70 con 0x7fc7b415a5e0 2026-02-24T18:40:43.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 5 monclient: authenticate success, global_id 5208 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 10 monclient: _renew_subs 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b41f4ca0 con 0x7fc7b415a5e0 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003e60 con 0x7fc7b415a5e0 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b0ff9640 10 monclient: dump: 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:43.754 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7b27fc640 4 set_mon_vals no callback set 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 10 client.5208.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 10 monclient: _renew_subs 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b414c9a0 con 0x7fc7b415a5e0 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.737+0000 7fc7ba4cd640 1 librados: init done 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.738+0000 7fc7ba4cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:43.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.738+0000 7fc7ba4cd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:43.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.738+0000 7fc7ba4cd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:43.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.738+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b4070300 con 0x7fc7b415a5e0 2026-02-24T18:40:43.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.843+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc7a80050d0 con 0x7fc7b415a5e0 2026-02-24T18:40:43.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.844+0000 7fc7b0ff9640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 0x7fc78407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:43.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.844+0000 7fc7b3fff640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 0x7fc78407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:43.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.844+0000 7fc7b3fff640 10 monclient: get_auth_request con 0x7fc784078e80 auth_method 0 2026-02-24T18:40:43.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.844+0000 7fc7b3fff640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 0x7fc78407b340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005980 tx=0x7fc7a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:43.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.844+0000 7fc7b0ff9640 10 client.5208.objecter ms_handle_connect 0x7fc784078e80 2026-02-24T18:40:43.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fc7a805e680 con 0x7fc7b415a5e0 2026-02-24T18:40:43.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 10 client.5208.objecter ms_dispatch2 0x7fc7b4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:43.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 3 client.5208.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:43.853 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 3 client.5208.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:43.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 20 client.5208.objecter dump_active .. 0 homeless 2026-02-24T18:40:43.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc7a8062640 con 0x7fc7b415a5e0 2026-02-24T18:40:43.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:43.854 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.851+0000 7fc7b0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:43.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.935+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc7b4157600 con 0x7fc784078e80 2026-02-24T18:40:43.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.936+0000 7fc7b0ff9640 1 -- 10.20.193.18:0/2566090600 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc7b4157600 con 0x7fc784078e80 2026-02-24T18:40:43.938 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 0 to deep-scrub 2026-02-24T18:40:43.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 10 librados: watch_flush enter 2026-02-24T18:40:43.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 10 librados: watch_flush exit 2026-02-24T18:40:43.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 20 client.5208.objecter shutdown clearing up homeless session... 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 10 client.5208.objecter successfully canceled tick 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 msgr2=0x7fc78407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 0x7fc78407b340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005980 tx=0x7fc7a0009040 comp rx=0 tx=0).stop 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 10 monclient: shutdown 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 msgr2=0x7fc7b41f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800cbb0 tx=0x7fc7a800a430 comp rx=0 tx=0).stop 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 shutdown_connections 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fc784078e80 0x7fc78407b340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fc7b415a5e0 0x7fc7b41f5760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fc7b40c99a0 0x7fc7b4072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 --2- 10.20.193.18:0/2566090600 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fc7b406ff20 0x7fc7b41f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:43.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 >> 10.20.193.18:0/2566090600 conn(0x7fc7b40cd780 msgr2=0x7fc7b414e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:43.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 shutdown_connections 2026-02-24T18:40:43.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 -- 10.20.193.18:0/2566090600 wait complete. 2026-02-24T18:40:43.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:43.937+0000 7fc7ba4cd640 1 librados: shutdown 2026-02-24T18:40:43.949 INFO:tasks.ceph:Scrubbing osd.4 2026-02-24T18:40:43.949 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:44.018 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 10 monclient: monmap: 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.018171+0000 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.018171+0000 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.019 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.020 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.023 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 1 Processor -- start 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 1 -- start start 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.017+0000 7f4a98658640 10 monclient: init 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient: _add_conns ranks=[4,5,3,0,1,2] 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4a901503e0 0x7f4a90159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-b con 0x7f4a901503e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4a9015a070 0x7f4a9006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-c con 0x7f4a9015a070 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4a900702f0 0x7f4a900c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-a con 0x7f4a900702f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f4a900c7d70 con 0x7f4a900702f0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4a900c8610 con 0x7f4a901503e0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4a90070760 con 0x7f4a9015a070 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a98658640 10 monclient(hunting): authenticate will time out at 1787.699339s 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4a900702f0 0x7f4a900c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a963cd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f4a901503e0 0x7f4a90159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a95bcc640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4a9015a070 0x7f4a9006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4a900702f0 0x7f4a900c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49310/0 (socket says 10.20.193.18:49310) 2026-02-24T18:40:44.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 1 -- 10.20.193.18:0/2564032118 learned_addr learned my addr 10.20.193.18:0/2564032118 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 10 monclient(hunting): get_auth_request con 0x7f4a900702f0 auth_method 0 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.018+0000 7f4a96bce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): get_auth_request con 0x7f4a9015a070 auth_method 0 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): get_auth_request con 0x7f4a901503e0 auth_method 0 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.019+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 10 monclient(hunting): handle_auth_done global_id 5211 payload 274 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 -- 10.20.193.18:0/2564032118 >> v2:10.20.193.110:3300/0 conn(0x7f4a9015a070 msgr2=0x7f4a9006fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 --2- 10.20.193.18:0/2564032118 >> v2:10.20.193.110:3300/0 conn(0x7f4a9015a070 0x7f4a9006fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 -- 10.20.193.18:0/2564032118 >> v2:10.20.193.32:3300/0 conn(0x7f4a901503e0 msgr2=0x7f4a90159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 --2- 10.20.193.18:0/2564032118 >> v2:10.20.193.32:3300/0 conn(0x7f4a901503e0 0x7f4a90159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 monclient: found mon.noname-a 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.020+0000 7f4a96bce640 1 -- 10.20.193.18:0/2564032118 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9015a570 con 0x7f4a900702f0 2026-02-24T18:40:44.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.021+0000 7f4a96bce640 1 --2- 10.20.193.18:0/2564032118 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a900702f0 0x7f4a900c8aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a900c8790 tx=0x7f4a8c02f2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be7bbb5260a813e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 1 -- 10.20.193.18:0/2564032118 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c02fd70 con 0x7f4a900702f0 2026-02-24T18:40:44.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:44.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: dump: 2026-02-24T18:40:44.034 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 5 monclient: authenticate success, global_id 5211 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 1 -- 10.20.193.18:0/2564032118 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0373f0 con 0x7f4a900702f0 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 1 -- 10.20.193.18:0/2564032118 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0377c0 con 0x7f4a900702f0 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 4 set_mon_vals no callback set 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a953cb640 10 monclient: dump: 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.036 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 10 monclient: shutdown 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 -- 10.20.193.18:0/2564032118 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a900702f0 msgr2=0x7f4a900c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 --2- 10.20.193.18:0/2564032118 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a900702f0 0x7f4a900c8aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a900c8790 tx=0x7f4a8c02f2c0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 -- 10.20.193.18:0/2564032118 shutdown_connections 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 --2- 10.20.193.18:0/2564032118 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a900702f0 0x7f4a900c8aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 --2- 10.20.193.18:0/2564032118 >> v2:10.20.193.110:3300/0 conn(0x7f4a9015a070 0x7f4a9006fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 --2- 10.20.193.18:0/2564032118 >> v2:10.20.193.32:3300/0 conn(0x7f4a901503e0 0x7f4a90159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 -- 10.20.193.18:0/2564032118 >> 10.20.193.18:0/2564032118 conn(0x7f4a900cd600 msgr2=0x7f4a900cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 -- 10.20.193.18:0/2564032118 shutdown_connections 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 -- 10.20.193.18:0/2564032118 wait complete. 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 10 monclient: monmap: 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:44.038 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 librados: starting msgr at 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.023+0000 7f4a98658640 1 librados: starting objecter 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 Processor -- start 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 -- start start 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 librados: setting wanted keys 2026-02-24T18:40:44.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 librados: calling monclient init 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient: init 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4a900702f0 0x7f4a901f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-c con 0x7f4a900702f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a901503e0 0x7f4a9014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-a con 0x7f4a901503e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4a9015a070 0x7f4a90073cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): picked mon.noname-b con 0x7f4a9015a070 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4a900c82a0 con 0x7f4a901503e0 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a900c7fc0 con 0x7f4a9015a070 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901507c0 con 0x7f4a900702f0 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a98658640 10 monclient(hunting): authenticate will time out at 1787.705487s 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a901503e0 0x7f4a9014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a963cd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4a900702f0 0x7f4a901f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a96bce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4a9015a070 0x7f4a90073cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a901503e0 0x7f4a9014eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49326/0 (socket says 10.20.193.18:49326) 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 1 -- 10.20.193.18:0/3898242704 learned_addr learned my addr 10.20.193.18:0/3898242704 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 10 monclient(hunting): get_auth_request con 0x7f4a901503e0 auth_method 0 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.024+0000 7f4a95bcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): get_auth_request con 0x7f4a900702f0 auth_method 0 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): get_auth_request con 0x7f4a9015a070 auth_method 0 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a95bcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a96bce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.025+0000 7f4a963cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 10 monclient(hunting): handle_auth_done global_id 5155 payload 824 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 -- 10.20.193.18:0/3898242704 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4a900702f0 msgr2=0x7f4a901f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4a900702f0 0x7f4a901f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 -- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a901503e0 msgr2=0x7f4a9014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4a901503e0 0x7f4a9014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 monclient: found mon.noname-b 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.026+0000 7f4a96bce640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a90072c20 con 0x7f4a9015a070 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.027+0000 7f4a96bce640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4a9015a070 0x7f4a90073cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c035000 tx=0x7f4a8c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.027+0000 7f4a76ffd640 10 client.?.objecter ms_handle_connect 0x7f4a9015a070 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.027+0000 7f4a76ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c046020 con 0x7f4a9015a070 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: dump: 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.044 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c041450 con 0x7f4a9015a070 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009510 con 0x7f4a9015a070 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a94bca640 4 set_mon_vals no callback set 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a76ffd640 10 monclient: dump: 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.046 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 5 monclient: authenticate success, global_id 5155 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 monclient: _renew_subs 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a90072f00 con 0x7f4a9015a070 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 client.5155.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 monclient: _renew_subs 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a900734c0 con 0x7f4a9015a070 2026-02-24T18:40:44.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 1 librados: init done 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 client.5155.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 15 client.5155.objecter _session_command_op_assign -1 1 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 20 client.5155.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 client.5155.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.029+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a900729d0 con 0x7f4a9015a070 2026-02-24T18:40:44.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.040+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4a8c041840 con 0x7f4a9015a070 2026-02-24T18:40:44.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4a60078e00 0x7f4a6007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a963cd640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4a60078e00 0x7f4a6007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a963cd640 10 monclient: get_auth_request con 0x7f4a60078e00 auth_method 0 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f4a8c0bd070 con 0x7f4a9015a070 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 10 client.5155.objecter ms_dispatch2 0x7f4a90000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 3 client.5155.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 3 client.5155.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 10 client.5155.objecter checking command 1 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7f4a600812e0 0x7f4a60083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter _get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 15 client.5155.objecter _session_command_op_remove -1 1 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter _get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 15 client.5155.objecter _session_command_op_assign 4 1 2026-02-24T18:40:44.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 4 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 10 client.5155.objecter _send_command 1 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a8c0373f0 con 0x7f4a600812e0 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.041+0000 7f4a76ffd640 20 client.5155.objecter dump_active .. 0 homeless 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a963cd640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4a60078e00 0x7f4a6007b2c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009bc0 tx=0x7f4a80009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a76ffd640 10 client.5155.objecter ms_handle_connect 0x7f4a60078e00 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a95bcc640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7f4a600812e0 0x7f4a60083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a95bcc640 10 monclient: get_auth_request con 0x7f4a600812e0 auth_method 0 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c083d00 con 0x7f4a9015a070 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.042+0000 7f4a76ffd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.043+0000 7f4a76ffd640 10 client.5155.objecter ms_handle_connect 0x7f4a600812e0 2026-02-24T18:40:44.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.043+0000 7f4a95bcc640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7f4a600812e0 0x7f4a60083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.054+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== osd.4 v2:10.20.193.18:6800/3481454300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4a8c0373f0 con 0x7f4a600812e0 2026-02-24T18:40:44.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.054+0000 7f4a76ffd640 10 client.5155.objecter ms_dispatch2 0x7f4a90000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:44.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.054+0000 7f4a76ffd640 10 client.5155.objecter _finish_command 1 = system:0 2026-02-24T18:40:44.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.054+0000 7f4a76ffd640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 4 2026-02-24T18:40:44.056 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.054+0000 7f4a76ffd640 15 client.5155.objecter _session_command_op_remove 4 1 2026-02-24T18:40:44.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 10 client.5155.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:44.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 15 client.5155.objecter _session_command_op_assign -1 2 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 20 client.5155.objecter _get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 20 client.5155.objecter _get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 15 client.5155.objecter _session_command_op_remove -1 2 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 20 client.5155.objecter get_session s=0x7f4a60080c10 osd=4 3 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 15 client.5155.objecter _session_command_op_assign 4 2 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 4 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 10 client.5155.objecter _send_command 2 2026-02-24T18:40:44.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.060+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 --> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a90159b30 con 0x7f4a600812e0 2026-02-24T18:40:44.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a76ffd640 1 -- 10.20.193.18:0/3898242704 <== osd.4 v2:10.20.193.18:6800/3481454300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a90159b30 con 0x7f4a600812e0 2026-02-24T18:40:44.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a76ffd640 10 client.5155.objecter ms_dispatch2 0x7f4a90000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:44.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a76ffd640 10 client.5155.objecter _finish_command 2 = system:0 2026-02-24T18:40:44.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a76ffd640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 4 2026-02-24T18:40:44.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a76ffd640 15 client.5155.objecter _session_command_op_remove 4 2 2026-02-24T18:40:44.069 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:44.069 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:44.069 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:44.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 10 librados: watch_flush enter 2026-02-24T18:40:44.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 10 librados: watch_flush exit 2026-02-24T18:40:44.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 10 client.5155.objecter close_session for osd.4 2026-02-24T18:40:44.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7f4a600812e0 msgr2=0x7f4a60083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.18:6800/3481454300,v1:10.20.193.18:6801/3481454300] conn(0x7f4a600812e0 0x7f4a60083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 20 client.5155.objecter put_session s=0x7f4a60080c10 osd=4 1 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 20 client.5155.objecter shutdown clearing up homeless session... 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 10 client.5155.objecter successfully canceled tick 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4a60078e00 msgr2=0x7f4a6007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4a60078e00 0x7f4a6007b2c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009bc0 tx=0x7f4a80009290 comp rx=0 tx=0).stop 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 10 monclient: shutdown 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4a9015a070 msgr2=0x7f4a90073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 --2- 10.20.193.18:0/3898242704 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4a9015a070 0x7f4a90073cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c035000 tx=0x7f4a8c004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a96bce640 1 -- 10.20.193.18:0/3898242704 reap_dead start 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 shutdown_connections 2026-02-24T18:40:44.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.066+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 >> 10.20.193.18:0/3898242704 conn(0x7f4a900cd600 msgr2=0x7f4a9006db60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.067+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 shutdown_connections 2026-02-24T18:40:44.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.067+0000 7f4a98658640 1 -- 10.20.193.18:0/3898242704 wait complete. 2026-02-24T18:40:44.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.067+0000 7f4a98658640 1 librados: shutdown 2026-02-24T18:40:44.077 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-24T18:40:44.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.144+0000 7fe57acbd640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.144+0000 7fe57acbd640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.144+0000 7fe57acbd640 10 monclient: monmap: 2026-02-24T18:40:44.146 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.145763+0000 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.145763+0000 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.147 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 Processor -- start 2026-02-24T18:40:44.150 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 -- start start 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient: init 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient: _add_conns ranks=[1,3,5,0,2,4] 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe5741527d0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe574152bb0 0x7fe5740c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-a con 0x7fe574152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe5740c5fc0 0x7fe5740c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-c con 0x7fe5740c5fc0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57406e300 con 0x7fe5741527d0 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.151 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe57406f1f0 con 0x7fe574152bb0 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe5740c6470 con 0x7fe5740c5fc0 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.145+0000 7fe57acbd640 10 monclient(hunting): authenticate will time out at 1787.826983s 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fe5740c5fc0 0x7fe5740c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe578a32640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fe5741527d0 0x7fe57406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34730/0 (socket says 10.20.193.18:34730) 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe578a32640 1 -- 10.20.193.18:0/1508603072 learned_addr learned my addr 10.20.193.18:0/1508603072 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe56bfff640 1 --2- 10.20.193.18:0/1508603072 >> v2:10.20.193.18:3300/0 conn(0x7fe574152bb0 0x7fe5740c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): get_auth_request con 0x7fe5740c5fc0 auth_method 0 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.152 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe56bfff640 10 monclient(hunting): get_auth_request con 0x7fe574152bb0 auth_method 0 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.146+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 10 monclient(hunting): handle_auth_done global_id 5144 payload 274 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 1 -- 10.20.193.18:0/1508603072 >> v2:10.20.193.18:3300/0 conn(0x7fe574152bb0 msgr2=0x7fe5740c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 1 --2- 10.20.193.18:0/1508603072 >> v2:10.20.193.18:3300/0 conn(0x7fe574152bb0 0x7fe5740c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 1 -- 10.20.193.18:0/1508603072 >> v1:10.20.193.32:6789/0 conn(0x7fe5741527d0 legacy=0x7fe57406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe579233640 1 -- 10.20.193.18:0/1508603072 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57406e5e0 con 0x7fe5740c5fc0 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe56b7fe640 1 -- 10.20.193.18:0/1508603072 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590392359 0 0) 0x7fe57406e300 con 0x7fe5741527d0 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.147+0000 7fe56b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-24T18:40:44.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.148+0000 7fe579233640 1 --2- 10.20.193.18:0/1508603072 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5740c5fc0 0x7fe5740c8e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe574158940 tx=0x7fe57002f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cadcb3a313bbb6b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 1 -- 10.20.193.18:0/1508603072 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57003c070 con 0x7fe5740c5fc0 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 10 monclient: dump: 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.161 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe56b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe57acbd640 5 monclient: authenticate success, global_id 5144 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.149+0000 7fe57acbd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 1 -- 10.20.193.18:0/1508603072 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe570037440 con 0x7fe5740c5fc0 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 1 -- 10.20.193.18:0/1508603072 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570037810 con 0x7fe5740c5fc0 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe56b7fe640 10 monclient: dump: 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:44.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 4 set_mon_vals no callback set 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 10 monclient: shutdown 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 -- 10.20.193.18:0/1508603072 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5740c5fc0 msgr2=0x7fe5740c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 --2- 10.20.193.18:0/1508603072 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5740c5fc0 0x7fe5740c8e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe574158940 tx=0x7fe57002f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 -- 10.20.193.18:0/1508603072 shutdown_connections 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 --2- 10.20.193.18:0/1508603072 >> v2:10.20.193.18:3300/0 conn(0x7fe574152bb0 0x7fe5740c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 --2- 10.20.193.18:0/1508603072 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5740c5fc0 0x7fe5740c8e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 -- 10.20.193.18:0/1508603072 >> 10.20.193.18:0/1508603072 conn(0x7fe5740cd780 msgr2=0x7fe5740cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 -- 10.20.193.18:0/1508603072 shutdown_connections 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.150+0000 7fe57acbd640 1 -- 10.20.193.18:0/1508603072 wait complete. 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient: monmap: 2026-02-24T18:40:44.164 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 librados: starting msgr at 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 librados: starting objecter 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 Processor -- start 2026-02-24T18:40:44.165 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 -- start start 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 librados: setting wanted keys 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 librados: calling monclient init 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient: init 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5740c5fc0 0x7fe574145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-a con 0x7fe5740c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-c con 0x7fe5741527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe574152bb0 0x7fe57414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): picked mon.noname-b con 0x7fe574152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.166 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe57406ee80 con 0x7fe5740c5fc0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe57406eba0 con 0x7fe574152bb0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740c93a0 con 0x7fe5741527d0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.151+0000 7fe57acbd640 10 monclient(hunting): authenticate will time out at 1787.832829s 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5740c5fc0 0x7fe574145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe574152bb0 0x7fe57414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37758/0 (socket says 10.20.193.18:37758) 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 1 -- 10.20.193.18:0/3231856172 learned_addr learned my addr 10.20.193.18:0/3231856172 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): get_auth_request con 0x7fe5741527d0 auth_method 0 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): get_auth_request con 0x7fe574152bb0 auth_method 0 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): get_auth_request con 0x7fe5740c5fc0 auth_method 0 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe56bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe578a32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.152+0000 7fe579233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 10 monclient(hunting): handle_auth_done global_id 5147 payload 824 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 -- 10.20.193.18:0/3231856172 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe574152bb0 msgr2=0x7fe57414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe574152bb0 0x7fe57414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 -- 10.20.193.18:0/3231856172 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5740c5fc0 msgr2=0x7fe574145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5740c5fc0 0x7fe574145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.153+0000 7fe56bfff640 1 -- 10.20.193.18:0/3231856172 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574157fa0 con 0x7fe5741527d0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.154+0000 7fe56bfff640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00e8c0 tx=0x7fe55c00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.154+0000 7fe5697fa640 10 client.?.objecter ms_handle_connect 0x7fe5741527d0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.154+0000 7fe5697fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c017070 con 0x7fe5741527d0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 10 monclient: dump: 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.170 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe5697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 5 monclient: authenticate success, global_id 5147 2026-02-24T18:40:44.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 10 monclient: _renew_subs 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57414cbf0 con 0x7fe5741527d0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 10 client.5147.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 10 monclient: _renew_subs 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57414d1b0 con 0x7fe5741527d0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.155+0000 7fe57acbd640 1 librados: init done 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c00c940 con 0x7fe5741527d0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c021060 con 0x7fe5741527d0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:44.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe5697fa640 10 monclient: dump: 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe56affd640 4 set_mon_vals no callback set 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe57acbd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe57acbd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe57acbd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.156+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5740c9730 con 0x7fe5741527d0 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.167+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe55c00ccf0 con 0x7fe5741527d0 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.167+0000 7fe5697fa640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 0x7fe54407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.168+0000 7fe578a32640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 0x7fe54407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.168+0000 7fe578a32640 10 monclient: get_auth_request con 0x7fe544078e00 auth_method 0 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.168+0000 7fe578a32640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 0x7fe54407b2c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe564007b40 tx=0x7fe564007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.168+0000 7fe5697fa640 10 client.5147.objecter ms_handle_connect 0x7fe544078e00 2026-02-24T18:40:44.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.261+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fe55c05e570 con 0x7fe5741527d0 2026-02-24T18:40:44.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.261+0000 7fe5697fa640 10 client.5147.objecter ms_dispatch2 0x7fe574000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:44.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.261+0000 7fe5697fa640 3 client.5147.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:44.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.261+0000 7fe5697fa640 3 client.5147.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:44.263 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.261+0000 7fe5697fa640 20 client.5147.objecter dump_active .. 0 homeless 2026-02-24T18:40:44.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.268+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe55c062530 con 0x7fe5741527d0 2026-02-24T18:40:44.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.268+0000 7fe5697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.269 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.268+0000 7fe5697fa640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:44.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.349+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe5741513a0 con 0x7fe544078e00 2026-02-24T18:40:44.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.349+0000 7fe5697fa640 1 -- 10.20.193.18:0/3231856172 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe5741513a0 con 0x7fe544078e00 2026-02-24T18:40:44.351 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 4 to deep-scrub 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 10 librados: watch_flush enter 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 10 librados: watch_flush exit 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 20 client.5147.objecter shutdown clearing up homeless session... 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 10 client.5147.objecter successfully canceled tick 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 msgr2=0x7fe54407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 0x7fe54407b2c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe564007b40 tx=0x7fe564007a90 comp rx=0 tx=0).stop 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 10 monclient: shutdown 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 msgr2=0x7fe57414d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.353 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00e8c0 tx=0x7fe55c00ed90 comp rx=0 tx=0).stop 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 shutdown_connections 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe544078e00 0x7fe54407b2c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe574152bb0 0x7fe57414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe5741527d0 0x7fe57414d9b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 --2- 10.20.193.18:0/3231856172 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe5740c5fc0 0x7fe574145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 >> 10.20.193.18:0/3231856172 conn(0x7fe5740cd780 msgr2=0x7fe57406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 shutdown_connections 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 -- 10.20.193.18:0/3231856172 wait complete. 2026-02-24T18:40:44.354 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.350+0000 7fe57acbd640 1 librados: shutdown 2026-02-24T18:40:44.360 INFO:tasks.ceph:Scrubbing osd.8 2026-02-24T18:40:44.360 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:44.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.432 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 10 monclient: monmap: 2026-02-24T18:40:44.432 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.432 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.431014+0000 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.431014+0000 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 1 Processor -- start 2026-02-24T18:40:44.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 1 -- start start 2026-02-24T18:40:44.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.430+0000 7fa0810ea640 10 monclient: init 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient: _add_conns ranks=[1,5,0,4,2,3] 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa07c155fe0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa07c1563c0 0x7fa07c074d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-c con 0x7fa07c1563c0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa07c0cc310 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07c0cca70 con 0x7fa07c0cc310 2026-02-24T18:40:44.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07c0ccd50 con 0x7fa07c155fe0 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa07c15cf30 con 0x7fa07c1563c0 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa0810ea640 10 monclient(hunting): authenticate will time out at 1788.112377s 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07a575640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa07c1563c0 0x7fa07c074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07b577640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa07c0cc310 0x7fa07c15d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48590/0 (socket says 10.20.193.18:48590) 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07b577640 1 -- 10.20.193.18:0/4254987832 learned_addr learned my addr 10.20.193.18:0/4254987832 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07a575640 10 monclient(hunting): get_auth_request con 0x7fa07c1563c0 auth_method 0 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07a575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.431+0000 7fa07a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.437 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient(hunting): handle_auth_done global_id 5150 payload 274 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 1 -- 10.20.193.18:0/4254987832 >> v1:10.20.193.32:6789/0 conn(0x7fa07c155fe0 legacy=0x7fa07c15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 1 -- 10.20.193.18:0/4254987832 >> v1:10.20.193.18:6789/0 conn(0x7fa07c0cc310 legacy=0x7fa07c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.432+0000 7fa07a575640 1 -- 10.20.193.18:0/4254987832 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c0cd030 con 0x7fa07c1563c0 2026-02-24T18:40:44.438 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.433+0000 7fa07a575640 1 --2- 10.20.193.18:0/4254987832 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c1563c0 0x7fa07c074d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098a0 tx=0x7fa06402ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568afe8fa4726359 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 1 -- 10.20.193.18:0/4254987832 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06403c070 con 0x7fa07c1563c0 2026-02-24T18:40:44.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:44.444 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: dump: 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.445 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 5 monclient: authenticate success, global_id 5150 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 1 -- 10.20.193.18:0/4254987832 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06402fb70 con 0x7fa07c1563c0 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 1 -- 10.20.193.18:0/4254987832 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064037440 con 0x7fa07c1563c0 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa079d74640 10 monclient: dump: 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.446 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 4 set_mon_vals no callback set 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 10 monclient: shutdown 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.447 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 1 -- 10.20.193.18:0/4254987832 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c1563c0 msgr2=0x7fa07c074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 1 --2- 10.20.193.18:0/4254987832 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c1563c0 0x7fa07c074d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098a0 tx=0x7fa06402ee60 comp rx=0 tx=0).stop 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 1 -- 10.20.193.18:0/4254987832 shutdown_connections 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 1 --2- 10.20.193.18:0/4254987832 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c1563c0 0x7fa07c074d60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.435+0000 7fa0810ea640 1 -- 10.20.193.18:0/4254987832 >> 10.20.193.18:0/4254987832 conn(0x7fa07c0ce830 msgr2=0x7fa07c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 -- 10.20.193.18:0/4254987832 shutdown_connections 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 -- 10.20.193.18:0/4254987832 wait complete. 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 10 monclient: monmap: 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.448 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 librados: starting msgr at 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 librados: starting objecter 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 Processor -- start 2026-02-24T18:40:44.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 -- start start 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 librados: setting wanted keys 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 1 librados: calling monclient init 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.436+0000 7fa0810ea640 10 monclient: init 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 0x7fa07c14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-c con 0x7fa07c0cc310 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa07c155fe0 0x7fa07c1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-b con 0x7fa07c155fe0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa07c1563c0 0x7fa07c1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): picked mon.noname-a con 0x7fa07c1563c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.450 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0cd5f0 con 0x7fa07c1563c0 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0cd960 con 0x7fa07c155fe0 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0cc760 con 0x7fa07c0cc310 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa0810ea640 10 monclient(hunting): authenticate will time out at 1788.118072s 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 0x7fa07c14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa07c1563c0 0x7fa07c1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07a575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa07c155fe0 0x7fa07c1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 0x7fa07c14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37780/0 (socket says 10.20.193.18:37780) 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 1 -- 10.20.193.18:0/3753081024 learned_addr learned my addr 10.20.193.18:0/3753081024 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.451 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): get_auth_request con 0x7fa07c0cc310 auth_method 0 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): get_auth_request con 0x7fa07c1563c0 auth_method 0 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07a575640 10 monclient(hunting): get_auth_request con 0x7fa07c155fe0 auth_method 0 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07a575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.437+0000 7fa07b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 10 monclient(hunting): handle_auth_done global_id 5153 payload 824 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 -- 10.20.193.18:0/3753081024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa07c155fe0 msgr2=0x7fa07c1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa07c155fe0 0x7fa07c1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 -- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa07c1563c0 msgr2=0x7fa07c1f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa07c1563c0 0x7fa07c1f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.438+0000 7fa07ad76640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c1f73c0 con 0x7fa07c0cc310 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.440+0000 7fa07ad76640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 0x7fa07c14baa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4b0 tx=0x7fa07000c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.440+0000 7fa05b7fe640 10 client.?.objecter ms_handle_connect 0x7fa07c0cc310 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.440+0000 7fa05b7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700052d0 con 0x7fa07c0cc310 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: dump: 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.454 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 5 monclient: authenticate success, global_id 5153 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 monclient: _renew_subs 2026-02-24T18:40:44.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c1f76a0 con 0x7fa07c0cc310 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 client.5153.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 monclient: _renew_subs 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c1f7c60 con 0x7fa07c0cc310 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 1 librados: init done 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa070005470 con 0x7fa07c0cc310 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700038f0 con 0x7fa07c0cc310 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa05b7fe640 10 monclient: dump: 2026-02-24T18:40:44.456 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa079573640 4 set_mon_vals no callback set 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 client.5153.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 15 client.5153.objecter _session_command_op_assign -1 1 2026-02-24T18:40:44.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 20 client.5153.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:44.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 client.5153.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.441+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa07c06be20 con 0x7fa07c0cc310 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.453+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa070003a90 con 0x7fa07c0cc310 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.453+0000 7fa05b7fe640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa048078e30 0x7fa04807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.454+0000 7fa07a575640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa048078e30 0x7fa04807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.454+0000 7fa07a575640 10 monclient: get_auth_request con 0x7fa048078e30 auth_method 0 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.454+0000 7fa05b7fe640 10 client.5153.objecter ms_handle_connect 0x7fa048078e30 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.454+0000 7fa07a575640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa048078e30 0x7fa04807b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa064039000 tx=0x7fa064033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa070097850 con 0x7fa07c0cc310 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 10 client.5153.objecter ms_dispatch2 0x7fa07c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 3 client.5153.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 3 client.5153.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 10 client.5153.objecter checking command 1 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7fa048081310 0x7fa048083730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter _get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 15 client.5153.objecter _session_command_op_remove -1 1 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter _get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 15 client.5153.objecter _session_command_op_assign 8 1 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 4 2026-02-24T18:40:44.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 10 client.5153.objecter _send_command 1 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa070010e50 con 0x7fa048081310 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 20 client.5153.objecter dump_active .. 0 homeless 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa07005e290 con 0x7fa07c0cc310 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa05b7fe640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa07b577640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7fa048081310 0x7fa048083730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.456+0000 7fa07b577640 10 monclient: get_auth_request con 0x7fa048081310 auth_method 0 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.458+0000 7fa07b577640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7fa048081310 0x7fa048083730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.458+0000 7fa05b7fe640 10 client.5153.objecter ms_handle_connect 0x7fa048081310 2026-02-24T18:40:44.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.468+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== osd.8 v2:10.20.193.18:6808/1977979333 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa070010e50 con 0x7fa048081310 2026-02-24T18:40:44.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.468+0000 7fa05b7fe640 10 client.5153.objecter ms_dispatch2 0x7fa07c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:44.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.468+0000 7fa05b7fe640 10 client.5153.objecter _finish_command 1 = system:0 2026-02-24T18:40:44.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.468+0000 7fa05b7fe640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 4 2026-02-24T18:40:44.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.468+0000 7fa05b7fe640 15 client.5153.objecter _session_command_op_remove 8 1 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 10 client.5153.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 15 client.5153.objecter _session_command_op_assign -1 2 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 20 client.5153.objecter _get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 20 client.5153.objecter _get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 15 client.5153.objecter _session_command_op_remove -1 2 2026-02-24T18:40:44.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 20 client.5153.objecter get_session s=0x7fa048080c40 osd=8 3 2026-02-24T18:40:44.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 15 client.5153.objecter _session_command_op_assign 8 2 2026-02-24T18:40:44.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 4 2026-02-24T18:40:44.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 10 client.5153.objecter _send_command 2 2026-02-24T18:40:44.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.474+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 --> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa07c06e5e0 con 0x7fa048081310 2026-02-24T18:40:44.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa05b7fe640 1 -- 10.20.193.18:0/3753081024 <== osd.8 v2:10.20.193.18:6808/1977979333 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa07c06e5e0 con 0x7fa048081310 2026-02-24T18:40:44.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa05b7fe640 10 client.5153.objecter ms_dispatch2 0x7fa07c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:44.482 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa05b7fe640 10 client.5153.objecter _finish_command 2 = system:0 2026-02-24T18:40:44.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa05b7fe640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 4 2026-02-24T18:40:44.483 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa05b7fe640 15 client.5153.objecter _session_command_op_remove 8 2 2026-02-24T18:40:44.483 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:44.483 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:44.483 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 10 librados: watch_flush enter 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 10 librados: watch_flush exit 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 10 client.5153.objecter close_session for osd.8 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7fa048081310 msgr2=0x7fa048083730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.18:6808/1977979333,v1:10.20.193.18:6809/1977979333] conn(0x7fa048081310 0x7fa048083730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 20 client.5153.objecter put_session s=0x7fa048080c40 osd=8 1 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 20 client.5153.objecter shutdown clearing up homeless session... 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 10 client.5153.objecter successfully canceled tick 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa048078e30 msgr2=0x7fa04807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.485 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa048078e30 0x7fa04807b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa064039000 tx=0x7fa064033000 comp rx=0 tx=0).stop 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 10 monclient: shutdown 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 msgr2=0x7fa07c14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 --2- 10.20.193.18:0/3753081024 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa07c0cc310 0x7fa07c14baa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4b0 tx=0x7fa07000c980 comp rx=0 tx=0).stop 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa07b577640 1 -- 10.20.193.18:0/3753081024 reap_dead start 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 shutdown_connections 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 >> 10.20.193.18:0/3753081024 conn(0x7fa07c0ce830 msgr2=0x7fa07c1545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 shutdown_connections 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 -- 10.20.193.18:0/3753081024 wait complete. 2026-02-24T18:40:44.486 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.481+0000 7fa0810ea640 1 librados: shutdown 2026-02-24T18:40:44.491 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-24T18:40:44.561 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 10 monclient: monmap: 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.561159+0000 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.561159+0000 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.562 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.563 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.563 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.563 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.563 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.563 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 1 Processor -- start 2026-02-24T18:40:44.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 1 -- start start 2026-02-24T18:40:44.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.560+0000 7f337577e640 10 monclient: init 2026-02-24T18:40:44.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient: _add_conns ranks=[4,2,1,3,5,0] 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f337006ff20 0x7f3370070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): picked mon.noname-b con 0x7f337006ff20 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f33700c99a0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f337015a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33700c7e90 con 0x7f337015a5e0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33700c8170 con 0x7f33700c99a0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3370159a40 con 0x7f337006ff20 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f337577e640 10 monclient(hunting): authenticate will time out at 1788.242295s 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336effd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f337006ff20 0x7f3370070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336e7fc640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f33700c99a0 0x7f3370159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:41764/0 (socket says 10.20.193.18:41764) 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336e7fc640 1 -- 10.20.193.18:0/1784733441 learned_addr learned my addr 10.20.193.18:0/1784733441 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336effd640 10 monclient(hunting): get_auth_request con 0x7f337006ff20 auth_method 0 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.561+0000 7f336effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.562+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.562+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.562+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 10 monclient(hunting): handle_auth_done global_id 5170 payload 274 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 1 -- 10.20.193.18:0/1784733441 >> v1:10.20.193.110:6789/0 conn(0x7f33700c99a0 legacy=0x7f3370159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 1 -- 10.20.193.18:0/1784733441 >> v1:10.20.193.32:6789/0 conn(0x7f337015a5e0 legacy=0x7f337014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 1 monclient: found mon.noname-b 2026-02-24T18:40:44.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.563+0000 7f336effd640 1 -- 10.20.193.18:0/1784733441 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33700c8450 con 0x7f337006ff20 2026-02-24T18:40:44.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.564+0000 7f336effd640 1 --2- 10.20.193.18:0/1784733441 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f3370070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33640098d0 tx=0x7f336402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e92e8eb43a89a0f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 1 -- 10.20.193.18:0/1784733441 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336403c070 con 0x7f337006ff20 2026-02-24T18:40:44.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: dump: 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.573 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 5 monclient: authenticate success, global_id 5170 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 1 -- 10.20.193.18:0/1784733441 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336402fb00 con 0x7f337006ff20 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 1 -- 10.20.193.18:0/1784733441 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364037440 con 0x7f337006ff20 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 4 set_mon_vals no callback set 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f336dffb640 10 monclient: dump: 2026-02-24T18:40:44.574 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 10 monclient: shutdown 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 -- 10.20.193.18:0/1784733441 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 msgr2=0x7f3370070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.575 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 --2- 10.20.193.18:0/1784733441 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f3370070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33640098d0 tx=0x7f336402ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 -- 10.20.193.18:0/1784733441 shutdown_connections 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 --2- 10.20.193.18:0/1784733441 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f3370070300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 -- 10.20.193.18:0/1784733441 >> 10.20.193.18:0/1784733441 conn(0x7f33700cd780 msgr2=0x7f33700cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 -- 10.20.193.18:0/1784733441 shutdown_connections 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 -- 10.20.193.18:0/1784733441 wait complete. 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 10 monclient: monmap: 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:44.576 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 librados: starting msgr at 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 librados: starting objecter 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.566+0000 7f337577e640 1 Processor -- start 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 -- start start 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 librados: setting wanted keys 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 librados: calling monclient init 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient: init 2026-02-24T18:40:44.577 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f33700c0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): picked mon.noname-b con 0x7f337006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 0x7f33700be420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): picked mon.noname-c con 0x7f33700c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 0x7f33700c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): picked mon.noname-a con 0x7f337015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3370150520 con 0x7f337015a5e0 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33700c8a10 con 0x7f337006ff20 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.578 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33701506a0 con 0x7f33700c99a0 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f337577e640 10 monclient(hunting): authenticate will time out at 1788.248422s 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f336f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 0x7f33700c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.567+0000 7f336e7fc640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 0x7f33700be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 0x7f33700c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49364/0 (socket says 10.20.193.18:49364) 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 1 -- 10.20.193.18:0/2497528011 learned_addr learned my addr 10.20.193.18:0/2497528011 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): get_auth_request con 0x7f337015a5e0 auth_method 0 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): get_auth_request con 0x7f33700c99a0 auth_method 0 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.579 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.568+0000 7f336effd640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f33700c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): get_auth_request con 0x7f337006ff20 auth_method 0 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.580 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 10 monclient(hunting): handle_auth_done global_id 5159 payload 824 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 -- 10.20.193.18:0/2497528011 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 msgr2=0x7f33700c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f33700c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 -- 10.20.193.18:0/2497528011 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 msgr2=0x7f33700c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 0x7f33700c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.569+0000 7f336e7fc640 1 -- 10.20.193.18:0/2497528011 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33700bedf0 con 0x7f33700c99a0 2026-02-24T18:40:44.581 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.571+0000 7f336e7fc640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 0x7f33700be420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f335800c530 tx=0x7f335800ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.571+0000 7f334f7fe640 10 client.?.objecter ms_handle_connect 0x7f33700c99a0 2026-02-24T18:40:44.582 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.571+0000 7f334f7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358017070 con 0x7f33700c99a0 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: dump: 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.584 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3358005de0 con 0x7f33700c99a0 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 5 monclient: authenticate success, global_id 5159 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 10 monclient: _renew_subs 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33700bf0d0 con 0x7f33700c99a0 2026-02-24T18:40:44.585 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33580072b0 con 0x7f33700c99a0 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f334f7fe640 10 monclient: dump: 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.586 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f336d7fa640 4 set_mon_vals no callback set 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 10 client.5159.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 10 monclient: _renew_subs 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33700c2400 con 0x7f33700c99a0 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.572+0000 7f337577e640 1 librados: init done 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.573+0000 7f337577e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.573+0000 7f337577e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.573+0000 7f337577e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.587 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.573+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3370070300 con 0x7f33700c99a0 2026-02-24T18:40:44.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.626+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3358007450 con 0x7f33700c99a0 2026-02-24T18:40:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.626+0000 7f334f7fe640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 0x7f333c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.627+0000 7f336effd640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 0x7f333c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.627+0000 7f336effd640 10 monclient: get_auth_request con 0x7f333c078e50 auth_method 0 2026-02-24T18:40:44.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.627+0000 7f336effd640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 0x7f333c07b310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3364039000 tx=0x7f3364033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.627+0000 7f334f7fe640 10 client.5159.objecter ms_handle_connect 0x7f333c078e50 2026-02-24T18:40:44.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.678+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f3358007770 con 0x7f33700c99a0 2026-02-24T18:40:44.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.678+0000 7f334f7fe640 10 client.5159.objecter ms_dispatch2 0x7f3370000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:44.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.678+0000 7f334f7fe640 3 client.5159.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:44.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.678+0000 7f334f7fe640 3 client.5159.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:44.680 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.678+0000 7f334f7fe640 20 client.5159.objecter dump_active .. 0 homeless 2026-02-24T18:40:44.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.685+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3358021020 con 0x7f33700c99a0 2026-02-24T18:40:44.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.685+0000 7f334f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.687 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.685+0000 7f334f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:44.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.770+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "8", "target": ["mon-mgr", ""]}) -- 0x7f33701573f0 con 0x7f333c078e50 2026-02-24T18:40:44.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.770+0000 7f334f7fe640 1 -- 10.20.193.18:0/2497528011 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 8 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f33701573f0 con 0x7f333c078e50 2026-02-24T18:40:44.772 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 8 to deep-scrub 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 10 librados: watch_flush enter 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 10 librados: watch_flush exit 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 20 client.5159.objecter shutdown clearing up homeless session... 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 10 client.5159.objecter successfully canceled tick 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 msgr2=0x7f333c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 0x7f333c07b310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3364039000 tx=0x7f3364033000 comp rx=0 tx=0).stop 2026-02-24T18:40:44.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 10 monclient: shutdown 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 msgr2=0x7f33700be420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 0x7f33700be420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f335800c530 tx=0x7f335800ca00 comp rx=0 tx=0).stop 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 shutdown_connections 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f333c078e50 0x7f333c07b310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f337015a5e0 0x7f33700c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f33700c99a0 0x7f33700be420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 --2- 10.20.193.18:0/2497528011 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f337006ff20 0x7f33700c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.771+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 >> 10.20.193.18:0/2497528011 conn(0x7f33700cd780 msgr2=0x7f3370159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.772+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 shutdown_connections 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.772+0000 7f337577e640 1 -- 10.20.193.18:0/2497528011 wait complete. 2026-02-24T18:40:44.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.772+0000 7f337577e640 1 librados: shutdown 2026-02-24T18:40:44.782 INFO:tasks.ceph:Scrubbing osd.12 2026-02-24T18:40:44.782 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:44.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.850+0000 7f57b151b640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.850+0000 7f57b151b640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.852 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.850+0000 7f57b151b640 10 monclient: monmap: 2026-02-24T18:40:44.852 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.852 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.851602+0000 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.851602+0000 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.853 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 1 Processor -- start 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 1 -- start start 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient: init 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient: _add_conns ranks=[2,0,1,5,3,4] 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f57ac0c9130 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57ac0c9510 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f57ac0c6340 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:44.856 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ac06e3b0 con 0x7f57ac0c9510 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ac06e690 con 0x7f57ac0c6340 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ac06e970 con 0x7f57ac0c9130 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.851+0000 7f57b151b640 10 monclient(hunting): authenticate will time out at 1788.532777s 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.852+0000 7f57aaffd640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f57ac0c9130 0x7f57ac06dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:41774/0 (socket says 10.20.193.18:41774) 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.852+0000 7f57aaffd640 1 -- 10.20.193.18:0/915105998 learned_addr learned my addr 10.20.193.18:0/915105998 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538193545 0 0) 0x7f57ac06e3b0 con 0x7f57ac0c9510 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): my global_id is 5232 2026-02-24T18:40:44.857 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5798003620 con 0x7f57ac0c9510 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968720638 0 0) 0x7f57ac06e970 con 0x7f57ac0c9130 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): my global_id is 5162 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ac06e3b0 con 0x7f57ac0c9130 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945724163 0 0) 0x7f57ac06e690 con 0x7f57ac0c6340 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 10 monclient(hunting): my global_id is 5176 2026-02-24T18:40:44.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.853+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ac06e970 con 0x7f57ac0c6340 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3928068862 0 0) 0x7f57ac06e3b0 con 0x7f57ac0c9130 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 >> v1:10.20.193.32:6789/0 conn(0x7f57ac0c6340 legacy=0x7f57ac15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 >> v1:10.20.193.18:6789/0 conn(0x7f57ac0c9510 legacy=0x7f57ac0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:40:44.867 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac06ec50 con 0x7f57ac0c9130 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7f57a0002ee0 con 0x7f57ac0c9130 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: dump: 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.868 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 >> v1:10.20.193.110:6789/0 conn(0x7f57ac0c9130 legacy=0x7f57ac06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:44.869 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5798003fb0 0x7f5798004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): picked mon.c con 0x7f5798003fb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57980044c0 0x7f5798004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): picked mon.b con 0x7f57980044c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f579800e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): picked mon.a con 0x7f579800bd40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f579800e7a0 con 0x7f579800bd40 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f579800e950 con 0x7f57980044c0 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f579800eb50 con 0x7f5798003fb0 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.870 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57a9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.855+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f579800e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57980044c0 0x7f5798004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5798003fb0 0x7f5798004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): get_auth_request con 0x7f579800bd40 auth_method 0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): get_auth_request con 0x7f5798003fb0 auth_method 0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): get_auth_request con 0x7f57980044c0 auth_method 0 2026-02-24T18:40:44.871 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.856+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_done global_id 5162 payload 293 2026-02-24T18:40:44.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 -- 10.20.193.18:0/915105998 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5798003fb0 msgr2=0x7f5798004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f5798003fb0 0x7f5798004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 -- 10.20.193.18:0/915105998 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57980044c0 msgr2=0x7f5798004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57980044c0 0x7f5798004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 monclient: found mon.a 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 -- 10.20.193.18:0/915105998 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac06e3b0 con 0x7f579800bd40 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57b151b640 5 monclient: authenticate success, global_id 5162 2026-02-24T18:40:44.873 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57b151b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.857+0000 7f57ab7fe640 1 -- 10.20.193.18:0/915105998 reap_dead start 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.858+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f579800e130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c320 tx=0x7f579c02f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b868a086d866465f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c003860 con 0x7f579800bd40 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: dump: 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.874 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c003c70 con 0x7f579800bd40 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 1 -- 10.20.193.18:0/915105998 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c005310 con 0x7f579800bd40 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 4 set_mon_vals no callback set 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57a9ffb640 10 monclient: dump: 2026-02-24T18:40:44.875 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 10 monclient: shutdown 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 -- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 msgr2=0x7f579800e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.876 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f579800e130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c320 tx=0x7f579c02f610 comp rx=0 tx=0).stop 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 -- 10.20.193.18:0/915105998 shutdown_connections 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 --2- 10.20.193.18:0/915105998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f579800e130 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 -- 10.20.193.18:0/915105998 >> 10.20.193.18:0/915105998 conn(0x7f57ac0cd600 msgr2=0x7f57ac0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 -- 10.20.193.18:0/915105998 shutdown_connections 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.860+0000 7f57b151b640 1 -- 10.20.193.18:0/915105998 wait complete. 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 10 monclient: monmap: 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.877 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 librados: starting msgr at 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 librados: starting objecter 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 Processor -- start 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 -- start start 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 librados: setting wanted keys 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 1 librados: calling monclient init 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.861+0000 7f57b151b640 10 monclient: init 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f57ac1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-a con 0x7f579800bd40 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57ac1f46d0 0x7f57ac1f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-b con 0x7f57ac1f46d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f57ac1f5900 0x7f57ac073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): picked mon.noname-c con 0x7f57ac1f5900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac06dba0 con 0x7f579800bd40 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac06f210 con 0x7f57ac1f46d0 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac06dd20 con 0x7f57ac1f5900 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57b151b640 10 monclient(hunting): authenticate will time out at 1788.543128s 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f57ac1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57ac1f46d0 0x7f57ac1f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57ab7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f57ac1f5900 0x7f57ac073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f57ac1477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49388/0 (socket says 10.20.193.18:49388) 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 1 -- 10.20.193.18:0/1944211892 learned_addr learned my addr 10.20.193.18:0/1944211892 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): get_auth_request con 0x7f579800bd40 auth_method 0 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): get_auth_request con 0x7f57ac1f46d0 auth_method 0 2026-02-24T18:40:44.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57ab7fe640 10 monclient(hunting): get_auth_request con 0x7f57ac1f5900 auth_method 0 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.862+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.863+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.863+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.863+0000 7f57ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 10 monclient(hunting): handle_auth_done global_id 5179 payload 824 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 -- 10.20.193.18:0/1944211892 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f57ac1f5900 msgr2=0x7f57ac073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f57ac1f5900 0x7f57ac073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 -- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 msgr2=0x7f57ac1477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f579800bd40 0x7f57ac1477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 monclient: found mon.noname-b 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.882 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.864+0000 7f57aa7fc640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac1f5d70 con 0x7f57ac1f46d0 2026-02-24T18:40:44.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.865+0000 7f57aa7fc640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57ac1f46d0 0x7f57ac1f53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f579400eaa0 tx=0x7f579400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.865+0000 7f578b7fe640 10 client.?.objecter ms_handle_connect 0x7f57ac1f46d0 2026-02-24T18:40:44.883 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.865+0000 7f578b7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:44.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794006ae0 con 0x7f57ac1f46d0 2026-02-24T18:40:44.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:44.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: dump: 2026-02-24T18:40:44.886 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:44.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 5 monclient: authenticate success, global_id 5179 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 monclient: _renew_subs 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac0c4220 con 0x7f57ac1f46d0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 client.5179.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 monclient: _renew_subs 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac0c3c60 con 0x7f57ac1f46d0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 1 librados: init done 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5794018070 con 0x7f57ac1f46d0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57a97fa640 4 set_mon_vals no callback set 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400c6c0 con 0x7f57ac1f46d0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:44.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f578b7fe640 10 monclient: dump: 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 client.5179.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 15 client.5179.objecter _session_command_op_assign -1 1 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 20 client.5179.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:44.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 client.5179.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.866+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57ac0c5c30 con 0x7f57ac1f46d0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.877+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f579400c860 con 0x7f57ac1f46d0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.878+0000 7f578b7fe640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5778078e00 0x7f577807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.878+0000 7f57aaffd640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5778078e00 0x7f577807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.878+0000 7f57aaffd640 10 monclient: get_auth_request con 0x7f5778078e00 auth_method 0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f5794097ef0 con 0x7f57ac1f46d0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 10 client.5179.objecter ms_dispatch2 0x7f57ac000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 3 client.5179.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 3 client.5179.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 10 client.5179.objecter checking command 1 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f5778081340 0x7f5778083760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter _get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 15 client.5179.objecter _session_command_op_remove -1 1 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter _get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 15 client.5179.objecter _session_command_op_assign 12 1 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 4 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 10 client.5179.objecter _send_command 1 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5794020e50 con 0x7f5778081340 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 20 client.5179.objecter dump_active .. 0 homeless 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f579405e930 con 0x7f57ac1f46d0 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f578b7fe640 10 client.5179.objecter ms_handle_connect 0x7f5778078e00 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f57aaffd640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5778078e00 0x7f577807b2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57a00028f0 tx=0x7f57a0003520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f5778081340 0x7f5778083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.879+0000 7f57ab7fe640 10 monclient: get_auth_request con 0x7f5778081340 auth_method 0 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.881+0000 7f57ab7fe640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f5778081340 0x7f5778083760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.881+0000 7f578b7fe640 10 client.5179.objecter ms_handle_connect 0x7f5778081340 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.890+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== osd.12 v2:10.20.193.18:6816/392025864 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5794020e50 con 0x7f5778081340 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.890+0000 7f578b7fe640 10 client.5179.objecter ms_dispatch2 0x7f57ac000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.890+0000 7f578b7fe640 10 client.5179.objecter _finish_command 1 = system:0 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.890+0000 7f578b7fe640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 4 2026-02-24T18:40:44.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.890+0000 7f578b7fe640 15 client.5179.objecter _session_command_op_remove 12 1 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 10 client.5179.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 15 client.5179.objecter _session_command_op_assign -1 2 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 20 client.5179.objecter _get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 20 client.5179.objecter _get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 15 client.5179.objecter _session_command_op_remove -1 2 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 20 client.5179.objecter get_session s=0x7f5778080c70 osd=12 3 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 15 client.5179.objecter _session_command_op_assign 12 2 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 4 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 10 client.5179.objecter _send_command 2 2026-02-24T18:40:44.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.896+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 --> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57ac074ed0 con 0x7f5778081340 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.909+0000 7f578b7fe640 1 -- 10.20.193.18:0/1944211892 <== osd.12 v2:10.20.193.18:6816/392025864 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7f57ac074ed0 con 0x7f5778081340 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.909+0000 7f578b7fe640 10 client.5179.objecter ms_dispatch2 0x7f57ac000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.909+0000 7f578b7fe640 10 client.5179.objecter _finish_command 2 = system:0 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.909+0000 7f578b7fe640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 4 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.909+0000 7f578b7fe640 15 client.5179.objecter _session_command_op_remove 12 2 2026-02-24T18:40:44.911 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:44.912 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:44.912 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 10 librados: watch_flush enter 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 10 librados: watch_flush exit 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 10 client.5179.objecter close_session for osd.12 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f5778081340 msgr2=0x7f5778083760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.18:6816/392025864,v1:10.20.193.18:6818/392025864] conn(0x7f5778081340 0x7f5778083760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 20 client.5179.objecter put_session s=0x7f5778080c70 osd=12 1 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 20 client.5179.objecter shutdown clearing up homeless session... 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 10 client.5179.objecter successfully canceled tick 2026-02-24T18:40:44.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5778078e00 msgr2=0x7f577807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f5778078e00 0x7f577807b2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57a00028f0 tx=0x7f57a0003520 comp rx=0 tx=0).stop 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 10 monclient: shutdown 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57ac1f46d0 msgr2=0x7f57ac1f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 --2- 10.20.193.18:0/1944211892 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f57ac1f46d0 0x7f57ac1f53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f579400eaa0 tx=0x7f579400ef70 comp rx=0 tx=0).stop 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57ab7fe640 1 -- 10.20.193.18:0/1944211892 reap_dead start 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 shutdown_connections 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 >> 10.20.193.18:0/1944211892 conn(0x7f57ac0cd600 msgr2=0x7f57ac14fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 shutdown_connections 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 -- 10.20.193.18:0/1944211892 wait complete. 2026-02-24T18:40:44.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.910+0000 7f57b151b640 1 librados: shutdown 2026-02-24T18:40:44.920 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2026-02-24T18:40:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.986+0000 7f2d53d96640 10 monclient: get_monmap_and_config 2026-02-24T18:40:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.986+0000 7f2d53d96640 10 monclient: build_initial_monmap 2026-02-24T18:40:44.988 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.986+0000 7f2d53d96640 10 monclient: monmap: 2026-02-24T18:40:44.988 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:44.988 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:44.987464+0000 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:44.987464+0000 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:44.989 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 Processor -- start 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 -- start start 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient: init 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient: _add_conns ranks=[0,5,4,1,2,3] 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d4c06ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f2d4c0c99a0 0x7f2d4c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-c con 0x7f2d4c0c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:44.993 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f2d4c15a5e0 0x7f2d4c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-b con 0x7f2d4c15a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d4c0c7e90 con 0x7f2d4c06ff20 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2d4c159a40 con 0x7f2d4c15a5e0 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2d4c0c8170 con 0x7f2d4c0c99a0 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.987+0000 7f2d53d96640 10 monclient(hunting): authenticate will time out at 1788.668666s 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d51b0b640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f2d4c06ff20 0x7f2d4c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48614/0 (socket says 10.20.193.18:48614) 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d51b0b640 1 -- 10.20.193.18:0/1794939618 learned_addr learned my addr 10.20.193.18:0/1794939618 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 1 --2- 10.20.193.18:0/1794939618 >> v2:10.20.193.110:3300/0 conn(0x7f2d4c0c99a0 0x7f2d4c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.994 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 1 --2- 10.20.193.18:0/1794939618 >> v2:10.20.193.32:3300/0 conn(0x7f2d4c15a5e0 0x7f2d4c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): get_auth_request con 0x7f2d4c0c99a0 auth_method 0 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): get_auth_request con 0x7f2d4c15a5e0 auth_method 0 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.995 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.988+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 1 -- 10.20.193.18:0/1794939618 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892433664 0 0) 0x7f2d4c0c7e90 con 0x7f2d4c06ff20 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 10 monclient(hunting): _negotiate 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 10 monclient(hunting): my global_id is 5238 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d50b09640 1 -- 10.20.193.18:0/1794939618 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d34003620 con 0x7f2d4c06ff20 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 10 monclient(hunting): handle_auth_done global_id 5168 payload 274 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 1 -- 10.20.193.18:0/1794939618 >> v2:10.20.193.32:3300/0 conn(0x7f2d4c15a5e0 msgr2=0x7f2d4c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 1 --2- 10.20.193.18:0/1794939618 >> v2:10.20.193.32:3300/0 conn(0x7f2d4c15a5e0 0x7f2d4c14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 1 -- 10.20.193.18:0/1794939618 >> v1:10.20.193.18:6789/0 conn(0x7f2d4c06ff20 legacy=0x7f2d4c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:44.996 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 10 monclient: _finish_hunting 0 2026-02-24T18:40:44.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 1 monclient: found mon.noname-c 2026-02-24T18:40:44.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:44.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:44.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.989+0000 7f2d5130a640 1 -- 10.20.193.18:0/1794939618 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c0c8450 con 0x7f2d4c0c99a0 2026-02-24T18:40:44.997 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.990+0000 7f2d5130a640 1 --2- 10.20.193.18:0/1794939618 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2d400098a0 tx=0x7f2d4002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38239b53f2c79f2a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.991+0000 7f2d50b09640 1 -- 10.20.193.18:0/1794939618 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4003c070 con 0x7f2d4c0c99a0 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.991+0000 7f2d50b09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.991+0000 7f2d50b09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.991+0000 7f2d50b09640 10 monclient: dump: 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.003 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 1 -- 10.20.193.18:0/1794939618 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4002fae0 con 0x7f2d4c0c99a0 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 5 monclient: authenticate success, global_id 5168 2026-02-24T18:40:45.004 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 1 -- 10.20.193.18:0/1794939618 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40037440 con 0x7f2d4c0c99a0 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d50b09640 10 monclient: dump: 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.005 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 4 set_mon_vals no callback set 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 10 monclient: shutdown 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 -- 10.20.193.18:0/1794939618 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c0c99a0 msgr2=0x7f2d4c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1794939618 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2d400098a0 tx=0x7f2d4002ee10 comp rx=0 tx=0).stop 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 -- 10.20.193.18:0/1794939618 shutdown_connections 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1794939618 >> v2:10.20.193.32:3300/0 conn(0x7f2d4c15a5e0 0x7f2d4c14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1794939618 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c070300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 -- 10.20.193.18:0/1794939618 >> 10.20.193.18:0/1794939618 conn(0x7f2d4c0cd780 msgr2=0x7f2d4c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 -- 10.20.193.18:0/1794939618 shutdown_connections 2026-02-24T18:40:45.006 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 1 -- 10.20.193.18:0/1794939618 wait complete. 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.992+0000 7f2d53d96640 10 monclient: monmap: 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.007 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 librados: starting msgr at 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 librados: starting objecter 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 Processor -- start 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 -- start start 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 librados: setting wanted keys 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 librados: calling monclient init 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient: init 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-a con 0x7f2d4c06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-b con 0x7f2d4c0c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:45.008 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c15a5e0 0x7f2d4c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): picked mon.noname-c con 0x7f2d4c15a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c150350 con 0x7f2d4c06ff20 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c0c8a10 con 0x7f2d4c0c99a0 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1504d0 con 0x7f2d4c15a5e0 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d53d96640 10 monclient(hunting): authenticate will time out at 1788.674661s 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d51b0b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.993+0000 7f2d5130a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.009 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c15a5e0 0x7f2d4c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49402/0 (socket says 10.20.193.18:49402) 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 1 -- 10.20.193.18:0/1816530497 learned_addr learned my addr 10.20.193.18:0/1816530497 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): get_auth_request con 0x7f2d4c06ff20 auth_method 0 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): get_auth_request con 0x7f2d4c0c99a0 auth_method 0 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): get_auth_request con 0x7f2d4c15a5e0 auth_method 0 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.010 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d51b0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5230c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.994+0000 7f2d5130a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 10 monclient(hunting): handle_auth_done global_id 5241 payload 824 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 -- 10.20.193.18:0/1816530497 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c15a5e0 msgr2=0x7f2d4c1f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.011 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c15a5e0 0x7f2d4c1f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 -- 10.20.193.18:0/1816530497 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d4c0c99a0 msgr2=0x7f2d4c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 monclient: found mon.noname-a 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.995+0000 7f2d51b0b640 1 -- 10.20.193.18:0/1816530497 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c1f3ff0 con 0x7f2d4c06ff20 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.996+0000 7f2d51b0b640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c730 tx=0x7f2d3c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.996+0000 7f2d3a7fc640 10 client.?.objecter ms_handle_connect 0x7f2d4c06ff20 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.996+0000 7f2d3a7fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c017070 con 0x7f2d4c06ff20 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:45.012 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: dump: 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.013 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 5 monclient: authenticate success, global_id 5241 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3c005bb0 con 0x7f2d4c06ff20 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c007070 con 0x7f2d4c06ff20 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 10 monclient: _renew_subs 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c155e20 con 0x7f2d4c06ff20 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3bfff640 4 set_mon_vals no callback set 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d3a7fc640 10 monclient: dump: 2026-02-24T18:40:45.014 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 10 client.5241.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 10 monclient: _renew_subs 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.015 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c1563e0 con 0x7f2d4c06ff20 2026-02-24T18:40:45.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.998+0000 7f2d53d96640 1 librados: init done 2026-02-24T18:40:45.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.999+0000 7f2d53d96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.999+0000 7f2d53d96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.999+0000 7f2d53d96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.016 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:44.999+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4c150860 con 0x7f2d4c06ff20 2026-02-24T18:40:45.108 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.106+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2d3c007210 con 0x7f2d4c06ff20 2026-02-24T18:40:45.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.107+0000 7f2d3a7fc640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 0x7f2d1c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.107+0000 7f2d5130a640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 0x7f2d1c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.107+0000 7f2d5130a640 10 monclient: get_auth_request con 0x7f2d1c078e80 auth_method 0 2026-02-24T18:40:45.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.107+0000 7f2d5130a640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 0x7f2d1c07b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d40039000 tx=0x7f2d40033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.109 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.107+0000 7f2d3a7fc640 10 client.5241.objecter ms_handle_connect 0x7f2d1c078e80 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f2d3c05d470 con 0x7f2d4c06ff20 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 10 client.5241.objecter ms_dispatch2 0x7f2d4c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 3 client.5241.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 3 client.5241.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 20 client.5241.objecter dump_active .. 0 homeless 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2d3c021020 con 0x7f2d4c06ff20 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.116 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.114+0000 7f2d3a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:45.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.197+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "12", "target": ["mon-mgr", ""]}) -- 0x7f2d4c0c7170 con 0x7f2d1c078e80 2026-02-24T18:40:45.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.197+0000 7f2d3a7fc640 1 -- 10.20.193.18:0/1816530497 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 12 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f2d4c0c7170 con 0x7f2d1c078e80 2026-02-24T18:40:45.199 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 12 to deep-scrub 2026-02-24T18:40:45.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 10 librados: watch_flush enter 2026-02-24T18:40:45.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 10 librados: watch_flush exit 2026-02-24T18:40:45.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 20 client.5241.objecter shutdown clearing up homeless session... 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 10 client.5241.objecter successfully canceled tick 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 msgr2=0x7f2d1c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 0x7f2d1c07b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d40039000 tx=0x7f2d40033000 comp rx=0 tx=0).stop 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 10 monclient: shutdown 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 msgr2=0x7f2d4c1f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c730 tx=0x7f2d3c00cc00 comp rx=0 tx=0).stop 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 shutdown_connections 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f2d1c078e80 0x7f2d1c07b340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f2d4c0c99a0 0x7f2d4c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f2d4c15a5e0 0x7f2d4c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 --2- 10.20.193.18:0/1816530497 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f2d4c06ff20 0x7f2d4c1f73c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 >> 10.20.193.18:0/1816530497 conn(0x7f2d4c0cd780 msgr2=0x7f2d4c14e620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 shutdown_connections 2026-02-24T18:40:45.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 -- 10.20.193.18:0/1816530497 wait complete. 2026-02-24T18:40:45.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.198+0000 7f2d53d96640 1 librados: shutdown 2026-02-24T18:40:45.208 INFO:tasks.ceph:Scrubbing osd.1 2026-02-24T18:40:45.209 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:45.278 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.276+0000 7f99f1a16640 10 monclient: get_monmap_and_config 2026-02-24T18:40:45.278 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.276+0000 7f99f1a16640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.278 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.276+0000 7f99f1a16640 10 monclient: monmap: 2026-02-24T18:40:45.278 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.278 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:45.277637+0000 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:45.277637+0000 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:45.279 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 Processor -- start 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 -- start start 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient: init 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient: _add_conns ranks=[3,1,0,2,4,5] 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f99ec1503e0 0x7f99ec159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-a con 0x7f99ec1503e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f99ec15a070 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f99ec0704c0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99ec15a570 con 0x7f99ec0704c0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99ec15a850 con 0x7f99ec15a070 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f99ec0c7d70 con 0x7f99ec1503e0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.277+0000 7f99f1a16640 10 monclient(hunting): authenticate will time out at 1788.958788s 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99eaffd640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f99ec1503e0 0x7f99ec159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99ea7fc640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f99ec15a070 0x7f99ec06fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34764/0 (socket says 10.20.193.18:34764) 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99ea7fc640 1 -- 10.20.193.18:0/1412951968 learned_addr learned my addr 10.20.193.18:0/1412951968 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99eaffd640 10 monclient(hunting): get_auth_request con 0x7f99ec1503e0 auth_method 0 2026-02-24T18:40:45.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.278+0000 7f99eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 1 -- 10.20.193.18:0/1412951968 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012631348 0 0) 0x7f99ec15a850 con 0x7f99ec15a070 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 10 monclient(hunting): _negotiate 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 10 monclient(hunting): my global_id is 5188 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.279+0000 7f99e9ffb640 1 -- 10.20.193.18:0/1412951968 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99d8003620 con 0x7f99ec15a070 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 10 monclient(hunting): handle_auth_done global_id 5244 payload 274 2026-02-24T18:40:45.284 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 1 -- 10.20.193.18:0/1412951968 >> v1:10.20.193.32:6789/0 conn(0x7f99ec15a070 legacy=0x7f99ec06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 1 -- 10.20.193.18:0/1412951968 >> v1:10.20.193.18:6789/0 conn(0x7f99ec0704c0 legacy=0x7f99ec0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 1 monclient: found mon.noname-a 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.285 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.280+0000 7f99eaffd640 1 -- 10.20.193.18:0/1412951968 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec0c7fc0 con 0x7f99ec1503e0 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.282+0000 7f99eaffd640 1 --2- 10.20.193.18:0/1412951968 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec159b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e002ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee441755b7dd3a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 1 -- 10.20.193.18:0/1412951968 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e003c070 con 0x7f99ec1503e0 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 10 monclient: dump: 2026-02-24T18:40:45.290 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.291 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.283+0000 7f99e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 1 -- 10.20.193.18:0/1412951968 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e002fc10 con 0x7f99ec1503e0 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 1 -- 10.20.193.18:0/1412951968 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0037440 con 0x7f99ec1503e0 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99e9ffb640 10 monclient: dump: 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.292 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 5 monclient: authenticate success, global_id 5244 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 4 set_mon_vals no callback set 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 10 monclient: shutdown 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 -- 10.20.193.18:0/1412951968 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 msgr2=0x7f99ec159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 --2- 10.20.193.18:0/1412951968 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec159b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e002ef00 comp rx=0 tx=0).stop 2026-02-24T18:40:45.293 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 -- 10.20.193.18:0/1412951968 shutdown_connections 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 --2- 10.20.193.18:0/1412951968 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec159b30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 -- 10.20.193.18:0/1412951968 >> 10.20.193.18:0/1412951968 conn(0x7f99ec0cd600 msgr2=0x7f99ec0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 -- 10.20.193.18:0/1412951968 shutdown_connections 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 1 -- 10.20.193.18:0/1412951968 wait complete. 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.284+0000 7f99f1a16640 10 monclient: monmap: 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:45.294 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 librados: starting msgr at 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 librados: starting objecter 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 Processor -- start 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 -- start start 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 librados: setting wanted keys 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 librados: calling monclient init 2026-02-24T18:40:45.295 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient: init 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99ec0704c0 0x7f99ec155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-b con 0x7f99ec0704c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-a con 0x7f99ec1503e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f99ec15a070 0x7f99ec0c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): picked mon.noname-c con 0x7f99ec15a070 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec1507c0 con 0x7f99ec1503e0 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec0c8610 con 0x7f99ec0704c0 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.296 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec150940 con 0x7f99ec15a070 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.285+0000 7f99f1a16640 10 monclient(hunting): authenticate will time out at 1788.966896s 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99ec0704c0 0x7f99ec155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec1f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49418/0 (socket says 10.20.193.18:49418) 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 1 -- 10.20.193.18:0/962656301 learned_addr learned my addr 10.20.193.18:0/962656301 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): get_auth_request con 0x7f99ec1503e0 auth_method 0 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.297 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eb7fe640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f99ec15a070 0x7f99ec0c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): get_auth_request con 0x7f99ec0704c0 auth_method 0 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eb7fe640 10 monclient(hunting): get_auth_request con 0x7f99ec15a070 auth_method 0 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.298 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.286+0000 7f99eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 10 monclient(hunting): handle_auth_done global_id 5250 payload 824 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 -- 10.20.193.18:0/962656301 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f99ec15a070 msgr2=0x7f99ec0c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f99ec15a070 0x7f99ec0c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 -- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99ec0704c0 msgr2=0x7f99ec155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f99ec0704c0 0x7f99ec155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 monclient: found mon.noname-a 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.299 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.287+0000 7f99ea7fc640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec155a50 con 0x7f99ec1503e0 2026-02-24T18:40:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.288+0000 7f99ea7fc640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec1f77f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99d400c4d0 tx=0x7f99d400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.288+0000 7f99c3fff640 10 client.?.objecter ms_handle_connect 0x7f99ec1503e0 2026-02-24T18:40:45.300 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.288+0000 7f99c3fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4017070 con 0x7f99ec1503e0 2026-02-24T18:40:45.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: dump: 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.302 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4005e00 con 0x7f99ec1503e0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 5 monclient: authenticate success, global_id 5250 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 monclient: _renew_subs 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec155d30 con 0x7f99ec1503e0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40072b0 con 0x7f99ec1503e0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99e97fa640 4 set_mon_vals no callback set 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99c3fff640 10 monclient: dump: 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.304 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 client.5250.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 monclient: _renew_subs 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec1562f0 con 0x7f99ec1503e0 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 1 librados: init done 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 client.5250.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 15 client.5250.objecter _session_command_op_assign -1 1 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 20 client.5250.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 client.5250.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:45.305 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.290+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99ec159b30 con 0x7f99ec1503e0 2026-02-24T18:40:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.302+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f99d4007450 con 0x7f99ec1503e0 2026-02-24T18:40:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.302+0000 7f99c3fff640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f99bc078e50 0x7f99bc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.303+0000 7f99eaffd640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f99bc078e50 0x7f99bc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.303+0000 7f99eaffd640 10 monclient: get_auth_request con 0x7f99bc078e50 auth_method 0 2026-02-24T18:40:45.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.303+0000 7f99eaffd640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f99bc078e50 0x7f99bc07b310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f99e0039000 tx=0x7f99e0033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.303+0000 7f99c3fff640 10 client.5250.objecter ms_handle_connect 0x7f99bc078e50 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f99d4007770 con 0x7f99ec1503e0 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 10 client.5250.objecter ms_dispatch2 0x7f99ec000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 3 client.5250.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 3 client.5250.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 10 client.5250.objecter checking command 1 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7f99bc081330 0x7f99bc083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter _get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 15 client.5250.objecter _session_command_op_remove -1 1 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter _get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 15 client.5250.objecter _session_command_op_assign 1 1 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 4 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 10 client.5250.objecter _send_command 1 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99d4020ec0 con 0x7f99bc081330 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.304+0000 7f99c3fff640 20 client.5250.objecter dump_active .. 0 homeless 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.305+0000 7f99eb7fe640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7f99bc081330 0x7f99bc083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.305+0000 7f99eb7fe640 10 monclient: get_auth_request con 0x7f99bc081330 auth_method 0 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.305+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99d405dae0 con 0x7f99ec1503e0 2026-02-24T18:40:45.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.305+0000 7f99c3fff640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.306+0000 7f99c3fff640 10 client.5250.objecter ms_handle_connect 0x7f99bc081330 2026-02-24T18:40:45.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.306+0000 7f99eb7fe640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7f99bc081330 0x7f99bc083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.317+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== osd.1 v2:10.20.193.32:6800/2795974188 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f99d4020ec0 con 0x7f99bc081330 2026-02-24T18:40:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.317+0000 7f99c3fff640 10 client.5250.objecter ms_dispatch2 0x7f99ec000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.317+0000 7f99c3fff640 10 client.5250.objecter _finish_command 1 = system:0 2026-02-24T18:40:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.317+0000 7f99c3fff640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 4 2026-02-24T18:40:45.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.317+0000 7f99c3fff640 15 client.5250.objecter _session_command_op_remove 1 1 2026-02-24T18:40:45.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 10 client.5250.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:45.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 15 client.5250.objecter _session_command_op_assign -1 2 2026-02-24T18:40:45.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 20 client.5250.objecter _get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 20 client.5250.objecter _get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 15 client.5250.objecter _session_command_op_remove -1 2 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 20 client.5250.objecter get_session s=0x7f99bc080c60 osd=1 3 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 15 client.5250.objecter _session_command_op_assign 1 2 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 4 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 10 client.5250.objecter _send_command 2 2026-02-24T18:40:45.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.322+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 --> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f99ec0c0cc0 con 0x7f99bc081330 2026-02-24T18:40:45.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99c3fff640 1 -- 10.20.193.18:0/962656301 <== osd.1 v2:10.20.193.32:6800/2795974188 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f99ec0c0cc0 con 0x7f99bc081330 2026-02-24T18:40:45.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99c3fff640 10 client.5250.objecter ms_dispatch2 0x7f99ec000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:45.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99c3fff640 10 client.5250.objecter _finish_command 2 = system:0 2026-02-24T18:40:45.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99c3fff640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 4 2026-02-24T18:40:45.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99c3fff640 15 client.5250.objecter _session_command_op_remove 1 2 2026-02-24T18:40:45.332 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:45.332 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:45.332 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:45.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 10 librados: watch_flush enter 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 10 librados: watch_flush exit 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 10 client.5250.objecter close_session for osd.1 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7f99bc081330 msgr2=0x7f99bc083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.32:6800/2795974188,v1:10.20.193.32:6801/2795974188] conn(0x7f99bc081330 0x7f99bc083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 20 client.5250.objecter put_session s=0x7f99bc080c60 osd=1 1 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 20 client.5250.objecter shutdown clearing up homeless session... 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 10 client.5250.objecter successfully canceled tick 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f99bc078e50 msgr2=0x7f99bc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f99bc078e50 0x7f99bc07b310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f99e0039000 tx=0x7f99e0033000 comp rx=0 tx=0).stop 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 10 monclient: shutdown 2026-02-24T18:40:45.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 msgr2=0x7f99ec1f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99f1a16640 1 --2- 10.20.193.18:0/962656301 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f99ec1503e0 0x7f99ec1f77f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99d400c4d0 tx=0x7f99d400c9a0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.329+0000 7f99eb7fe640 1 -- 10.20.193.18:0/962656301 reap_dead start 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.330+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 shutdown_connections 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.330+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 >> 10.20.193.18:0/962656301 conn(0x7f99ec0cd600 msgr2=0x7f99ec14f200 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.330+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 shutdown_connections 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.330+0000 7f99f1a16640 1 -- 10.20.193.18:0/962656301 wait complete. 2026-02-24T18:40:45.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.330+0000 7f99f1a16640 1 librados: shutdown 2026-02-24T18:40:45.340 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-24T18:40:45.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: get_monmap_and_config 2026-02-24T18:40:45.409 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: monmap: 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:45.408895+0000 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:45.408895+0000 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:45.410 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:45.411 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:45.411 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:45.411 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 1 Processor -- start 2026-02-24T18:40:45.413 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 1 -- start start 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: init 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient: _add_conns ranks=[0,5,2,3,4,1] 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f49d006ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f49d00c99a0 0x7f49d0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.408+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-c con 0x7f49d00c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f49d015a5e0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d00c7e90 con 0x7f49d006ff20 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d00c8170 con 0x7f49d015a5e0 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.414 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f49d0159a40 con 0x7f49d00c99a0 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49d4a3c640 10 monclient(hunting): authenticate will time out at 1789.090023s 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49ce575640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f49d006ff20 0x7f49d0159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48634/0 (socket says 10.20.193.18:48634) 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49ce575640 1 -- 10.20.193.18:0/227728269 learned_addr learned my addr 10.20.193.18:0/227728269 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49cdd74640 1 --2- 10.20.193.18:0/227728269 >> v2:10.20.193.110:3300/0 conn(0x7f49d00c99a0 0x7f49d0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49cdd74640 10 monclient(hunting): get_auth_request con 0x7f49d00c99a0 auth_method 0 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49cdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49cdd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.409+0000 7f49cdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.415 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 1 -- 10.20.193.18:0/227728269 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195485222 0 0) 0x7f49d00c7e90 con 0x7f49d006ff20 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 10 monclient(hunting): _negotiate 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 10 monclient(hunting): my global_id is 5253 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cd573640 1 -- 10.20.193.18:0/227728269 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49bc003620 con 0x7f49d006ff20 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient(hunting): handle_auth_done global_id 5177 payload 274 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 1 -- 10.20.193.18:0/227728269 >> v1:10.20.193.110:6789/0 conn(0x7f49d015a5e0 legacy=0x7f49d014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 1 -- 10.20.193.18:0/227728269 >> v1:10.20.193.18:6789/0 conn(0x7f49d006ff20 legacy=0x7f49d0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 1 monclient: found mon.noname-c 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.416 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:45.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.410+0000 7f49cdd74640 1 -- 10.20.193.18:0/227728269 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d00c8450 con 0x7f49d00c99a0 2026-02-24T18:40:45.417 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.412+0000 7f49cdd74640 1 --2- 10.20.193.18:0/227728269 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d00c99a0 0x7f49d0070300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098d0 tx=0x7f49b802ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4de0359e4190fe server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 1 -- 10.20.193.18:0/227728269 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b803c070 con 0x7f49d00c99a0 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: dump: 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.421 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.422 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 1 -- 10.20.193.18:0/227728269 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b802fb00 con 0x7f49d00c99a0 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 1 -- 10.20.193.18:0/227728269 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8037440 con 0x7f49d00c99a0 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49cd573640 10 monclient: dump: 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.423 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 5 monclient: authenticate success, global_id 5177 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 4 set_mon_vals no callback set 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 10 monclient: shutdown 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 -- 10.20.193.18:0/227728269 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d00c99a0 msgr2=0x7f49d0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/227728269 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d00c99a0 0x7f49d0070300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098d0 tx=0x7f49b802ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 -- 10.20.193.18:0/227728269 shutdown_connections 2026-02-24T18:40:45.424 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/227728269 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d00c99a0 0x7f49d0070300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 -- 10.20.193.18:0/227728269 >> 10.20.193.18:0/227728269 conn(0x7f49d00cd780 msgr2=0x7f49d00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 -- 10.20.193.18:0/227728269 shutdown_connections 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.414+0000 7f49d4a3c640 1 -- 10.20.193.18:0/227728269 wait complete. 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient: monmap: 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.425 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 librados: starting msgr at 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 librados: starting objecter 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 Processor -- start 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 -- start start 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 librados: setting wanted keys 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 librados: calling monclient init 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient: init 2026-02-24T18:40:45.426 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 0x7f49d00730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-a con 0x7f49d006ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f49d00c99a0 0x7f49d006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-b con 0x7f49d00c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 0x7f49d01e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): picked mon.noname-c con 0x7f49d015a5e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0150520 con 0x7f49d006ff20 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00c8a10 con 0x7f49d00c99a0 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.427 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01506a0 con 0x7f49d015a5e0 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.415+0000 7f49d4a3c640 10 monclient(hunting): authenticate will time out at 1789.096811s 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 0x7f49d00730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 0x7f49d01e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 0x7f49d00730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49426/0 (socket says 10.20.193.18:49426) 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 1 -- 10.20.193.18:0/4210958173 learned_addr learned my addr 10.20.193.18:0/4210958173 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): get_auth_request con 0x7f49d006ff20 auth_method 0 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f49d00c99a0 0x7f49d006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.428 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): get_auth_request con 0x7f49d015a5e0 auth_method 0 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): get_auth_request con 0x7f49d00c99a0 auth_method 0 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ce575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.429 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49ced76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.416+0000 7f49cdd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 10 monclient(hunting): handle_auth_done global_id 5180 payload 824 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 -- 10.20.193.18:0/4210958173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f49d00c99a0 msgr2=0x7f49d006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f49d00c99a0 0x7f49d006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 -- 10.20.193.18:0/4210958173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 msgr2=0x7f49d00730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 0x7f49d00730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 monclient: found mon.noname-c 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:45.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.417+0000 7f49ced76640 1 -- 10.20.193.18:0/4210958173 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d0155050 con 0x7f49d015a5e0 2026-02-24T18:40:45.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.418+0000 7f49ced76640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 0x7f49d01e68f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49c000be50 tx=0x7f49c000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.418+0000 7f49b6ffd640 10 client.?.objecter ms_handle_connect 0x7f49d015a5e0 2026-02-24T18:40:45.431 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.418+0000 7f49b6ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0002a60 con 0x7f49d015a5e0 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: dump: 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.433 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0010070 con 0x7f49d015a5e0 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0002e40 con 0x7f49d015a5e0 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.434 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49b6ffd640 10 monclient: dump: 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49ccd72640 4 set_mon_vals no callback set 2026-02-24T18:40:45.435 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 5 monclient: authenticate success, global_id 5180 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 10 monclient: _renew_subs 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0155330 con 0x7f49d015a5e0 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 10 client.5180.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 10 monclient: _renew_subs 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01e60f0 con 0x7f49d015a5e0 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.420+0000 7f49d4a3c640 1 librados: init done 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.421+0000 7f49d4a3c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.421+0000 7f49d4a3c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.421+0000 7f49d4a3c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:45.436 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.421+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d01573f0 con 0x7f49d015a5e0 2026-02-24T18:40:45.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.523+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f49c0004070 con 0x7f49d015a5e0 2026-02-24T18:40:45.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.524+0000 7f49b6ffd640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 0x7f499c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.524+0000 7f49ce575640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 0x7f499c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.524+0000 7f49ce575640 10 monclient: get_auth_request con 0x7f499c078e00 auth_method 0 2026-02-24T18:40:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.525+0000 7f49ce575640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 0x7f499c07b2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005980 tx=0x7f49c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.525+0000 7f49b6ffd640 10 client.5180.objecter ms_handle_connect 0x7f499c078e00 2026-02-24T18:40:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.526+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f49c005dc10 con 0x7f49d015a5e0 2026-02-24T18:40:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.526+0000 7f49b6ffd640 10 client.5180.objecter ms_dispatch2 0x7f49d0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.526+0000 7f49b6ffd640 3 client.5180.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.526+0000 7f49b6ffd640 3 client.5180.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:45.528 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.526+0000 7f49b6ffd640 20 client.5180.objecter dump_active .. 0 homeless 2026-02-24T18:40:45.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.532+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f49c0061bd0 con 0x7f49d015a5e0 2026-02-24T18:40:45.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.532+0000 7f49b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.534 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.532+0000 7f49b6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:45.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.616+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f49d01500b0 con 0x7f499c078e00 2026-02-24T18:40:45.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.616+0000 7f49b6ffd640 1 -- 10.20.193.18:0/4210958173 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f49d01500b0 con 0x7f499c078e00 2026-02-24T18:40:45.618 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 1 to deep-scrub 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 10 librados: watch_flush enter 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 10 librados: watch_flush exit 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 20 client.5180.objecter shutdown clearing up homeless session... 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 10 client.5180.objecter successfully canceled tick 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 msgr2=0x7f499c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 0x7f499c07b2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005980 tx=0x7f49c4009040 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 10 monclient: shutdown 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 msgr2=0x7f49d01e68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 0x7f49d01e68f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49c000be50 tx=0x7f49c000bf50 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 shutdown_connections 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f499c078e00 0x7f499c07b2c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.617+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f49d00c99a0 0x7f49d006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f49d015a5e0 0x7f49d01e68f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 --2- 10.20.193.18:0/4210958173 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f49d006ff20 0x7f49d00730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 >> 10.20.193.18:0/4210958173 conn(0x7f49d00cd780 msgr2=0x7f49d00c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 shutdown_connections 2026-02-24T18:40:45.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 -- 10.20.193.18:0/4210958173 wait complete. 2026-02-24T18:40:45.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.618+0000 7f49d4a3c640 1 librados: shutdown 2026-02-24T18:40:45.628 INFO:tasks.ceph:Scrubbing osd.5 2026-02-24T18:40:45.628 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.699+0000 7fa1649ce640 10 monclient: get_monmap_and_config 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.699+0000 7fa1649ce640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.699+0000 7fa1649ce640 10 monclient: monmap: 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:45.700680+0000 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:45.700680+0000 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:45.701 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.700+0000 7fa1649ce640 1 Processor -- start 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.700+0000 7fa1649ce640 1 -- start start 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.700+0000 7fa1649ce640 10 monclient: init 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.700+0000 7fa1649ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.700+0000 7fa1649ce640 10 monclient: _add_conns ranks=[0,3,4,2,1,5] 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa15c155e50 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa15c06e690 0x7fa15c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-a con 0x7fa15c06e690 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa15c15c740 0x7fa15c0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-b con 0x7fa15c15c740 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c15cbf0 con 0x7fa15c155e50 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa15c070150 con 0x7fa15c06e690 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa15c06f540 con 0x7fa15c15c740 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.702 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa1649ce640 10 monclient(hunting): authenticate will time out at 1789.382053s 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa161f42640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa15c06e690 0x7fa15c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162f44640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fa15c15c740 0x7fa15c0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162743640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa15c155e50 0x7fa15c15f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48648/0 (socket says 10.20.193.18:48648) 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162743640 1 -- 10.20.193.18:0/3649897333 learned_addr learned my addr 10.20.193.18:0/3649897333 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162f44640 10 monclient(hunting): get_auth_request con 0x7fa15c15c740 auth_method 0 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162f44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162f44640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa162f44640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa161f42640 10 monclient(hunting): get_auth_request con 0x7fa15c06e690 auth_method 0 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa161f42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa161f42640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.701+0000 7fa161f42640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.703 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.702+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 10 monclient(hunting): handle_auth_done global_id 5197 payload 274 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 1 -- 10.20.193.18:0/3649897333 >> v2:10.20.193.18:3300/0 conn(0x7fa15c06e690 msgr2=0x7fa15c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 1 --2- 10.20.193.18:0/3649897333 >> v2:10.20.193.18:3300/0 conn(0x7fa15c06e690 0x7fa15c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 1 -- 10.20.193.18:0/3649897333 >> v1:10.20.193.18:6789/0 conn(0x7fa15c155e50 legacy=0x7fa15c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 1 monclient: found mon.noname-b 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.704 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.703+0000 7fa162f44640 1 -- 10.20.193.18:0/3649897333 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c06f820 con 0x7fa15c15c740 2026-02-24T18:40:45.705 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.704+0000 7fa162f44640 1 --2- 10.20.193.18:0/3649897333 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c15c740 0x7fa15c0705e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa15c073f90 tx=0x7fa15002f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6352e48320e6c4fe server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 1 -- 10.20.193.18:0/3649897333 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15003c070 con 0x7fa15c15c740 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: dump: 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 1 -- 10.20.193.18:0/3649897333 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa150037440 con 0x7fa15c15c740 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 1 -- 10.20.193.18:0/3649897333 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150037810 con 0x7fa15c15c740 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 5 monclient: authenticate success, global_id 5197 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:45.707 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 4 set_mon_vals no callback set 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa161741640 10 monclient: dump: 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.708 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 10 monclient: shutdown 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 -- 10.20.193.18:0/3649897333 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c15c740 msgr2=0x7fa15c0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 --2- 10.20.193.18:0/3649897333 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c15c740 0x7fa15c0705e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa15c073f90 tx=0x7fa15002f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 -- 10.20.193.18:0/3649897333 shutdown_connections 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 --2- 10.20.193.18:0/3649897333 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c15c740 0x7fa15c0705e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 --2- 10.20.193.18:0/3649897333 >> v2:10.20.193.18:3300/0 conn(0x7fa15c06e690 0x7fa15c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 -- 10.20.193.18:0/3649897333 >> 10.20.193.18:0/3649897333 conn(0x7fa15c0cc0a0 msgr2=0x7fa15c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 -- 10.20.193.18:0/3649897333 shutdown_connections 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 1 -- 10.20.193.18:0/3649897333 wait complete. 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.706+0000 7fa1649ce640 10 monclient: monmap: 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 librados: starting msgr at 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 librados: starting objecter 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 Processor -- start 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 -- start start 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 librados: setting wanted keys 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 librados: calling monclient init 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient: init 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 0x7fa15c1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-a con 0x7fa15c155e50 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa15c15c740 0x7fa15c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-c con 0x7fa15c15c740 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c0ce6c0 0x7fa15c0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): picked mon.noname-b con 0x7fa15c0ce6c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c156230 con 0x7fa15c155e50 2026-02-24T18:40:45.709 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c06fde0 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1563b0 con 0x7fa15c15c740 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa1649ce640 10 monclient(hunting): authenticate will time out at 1789.388522s 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa162743640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 0x7fa15c1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.707+0000 7fa162f44640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c0ce6c0 0x7fa15c0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 0x7fa15c1f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49444/0 (socket says 10.20.193.18:49444) 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 1 -- 10.20.193.18:0/4275683393 learned_addr learned my addr 10.20.193.18:0/4275683393 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): get_auth_request con 0x7fa15c155e50 auth_method 0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa15c15c740 0x7fa15c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): get_auth_request con 0x7fa15c0ce6c0 auth_method 0 2026-02-24T18:40:45.710 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): get_auth_request con 0x7fa15c15c740 auth_method 0 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162743640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa162f44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.708+0000 7fa161f42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 10 monclient(hunting): handle_auth_done global_id 5200 payload 824 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 -- 10.20.193.18:0/4275683393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa15c15c740 msgr2=0x7fa15c154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa15c15c740 0x7fa15c154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 -- 10.20.193.18:0/4275683393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 msgr2=0x7fa15c1f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 0x7fa15c1f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 monclient: found mon.noname-b 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162f44640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1f3930 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.711 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.709+0000 7fa162743640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa15c155e50 0x7fa15c1f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:40:45.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.711+0000 7fa162f44640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c0ce6c0 0x7fa15c0c76a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa150036000 tx=0x7fa150004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.711+0000 7fa146ffd640 10 client.?.objecter ms_handle_connect 0x7fa15c0ce6c0 2026-02-24T18:40:45.712 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.711+0000 7fa146ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:45.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15003c050 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.713 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: dump: 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 5 monclient: authenticate success, global_id 5200 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 monclient: _renew_subs 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c1f3c10 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 client.5200.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 monclient: _renew_subs 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c1f41d0 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 1 librados: init done 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa150048070 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa160f40640 4 set_mon_vals no callback set 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150041a10 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa146ffd640 10 monclient: dump: 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.714 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 client.5200.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 15 client.5200.objecter _session_command_op_assign -1 1 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 20 client.5200.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 client.5200.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.715 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.712+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa15c1f7a90 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.723+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa150041bb0 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.724+0000 7fa146ffd640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa12c078e00 0x7fa12c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.724+0000 7fa162743640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa12c078e00 0x7fa12c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.724+0000 7fa162743640 10 monclient: get_auth_request con 0x7fa12c078e00 auth_method 0 2026-02-24T18:40:45.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.725+0000 7fa162743640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa12c078e00 0x7fa12c07b2c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa154006f80 tx=0x7fa1540029b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.726 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.725+0000 7fa146ffd640 10 client.5200.objecter ms_handle_connect 0x7fa12c078e00 2026-02-24T18:40:45.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa1500bdce0 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 10 client.5200.objecter ms_dispatch2 0x7fa15c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:45.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 3 client.5200.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:45.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 3 client.5200.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:45.727 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 10 client.5200.objecter checking command 1 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7fa12c0812e0 0x7fa12c083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter _get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 15 client.5200.objecter _session_command_op_remove -1 1 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter _get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 15 client.5200.objecter _session_command_op_assign 5 1 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 4 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 10 client.5200.objecter _send_command 1 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa150037440 con 0x7fa12c0812e0 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 20 client.5200.objecter dump_active .. 0 homeless 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa150084950 con 0x7fa15c0ce6c0 2026-02-24T18:40:45.728 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.726+0000 7fa146ffd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:45.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.769+0000 7fa161f42640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7fa12c0812e0 0x7fa12c083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.772+0000 7fa161f42640 10 monclient: get_auth_request con 0x7fa12c0812e0 auth_method 0 2026-02-24T18:40:45.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.805+0000 7fa146ffd640 10 client.5200.objecter ms_handle_connect 0x7fa12c0812e0 2026-02-24T18:40:45.806 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.805+0000 7fa161f42640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7fa12c0812e0 0x7fa12c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.815+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== osd.5 v2:10.20.193.32:6808/2128834550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa150037440 con 0x7fa12c0812e0 2026-02-24T18:40:45.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.815+0000 7fa146ffd640 10 client.5200.objecter ms_dispatch2 0x7fa15c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:45.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.815+0000 7fa146ffd640 10 client.5200.objecter _finish_command 1 = system:0 2026-02-24T18:40:45.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.815+0000 7fa146ffd640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 4 2026-02-24T18:40:45.817 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.815+0000 7fa146ffd640 15 client.5200.objecter _session_command_op_remove 5 1 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 10 client.5200.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 15 client.5200.objecter _session_command_op_assign -1 2 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 20 client.5200.objecter _get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 20 client.5200.objecter _get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 15 client.5200.objecter _session_command_op_remove -1 2 2026-02-24T18:40:45.823 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 20 client.5200.objecter get_session s=0x7fa12c080c10 osd=5 3 2026-02-24T18:40:45.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 15 client.5200.objecter _session_command_op_assign 5 2 2026-02-24T18:40:45.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 4 2026-02-24T18:40:45.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 10 client.5200.objecter _send_command 2 2026-02-24T18:40:45.824 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.821+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 --> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa15c06c820 con 0x7fa12c0812e0 2026-02-24T18:40:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa146ffd640 1 -- 10.20.193.18:0/4275683393 <== osd.5 v2:10.20.193.32:6808/2128834550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7fa15c06c820 con 0x7fa12c0812e0 2026-02-24T18:40:45.836 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa146ffd640 10 client.5200.objecter ms_dispatch2 0x7fa15c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa146ffd640 10 client.5200.objecter _finish_command 2 = system:0 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa146ffd640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 4 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa146ffd640 15 client.5200.objecter _session_command_op_remove 5 2 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:45.837 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 10 librados: watch_flush enter 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 10 librados: watch_flush exit 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 10 client.5200.objecter close_session for osd.5 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7fa12c0812e0 msgr2=0x7fa12c083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:6808/2128834550,v1:10.20.193.32:6809/2128834550] conn(0x7fa12c0812e0 0x7fa12c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 20 client.5200.objecter put_session s=0x7fa12c080c10 osd=5 1 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 20 client.5200.objecter shutdown clearing up homeless session... 2026-02-24T18:40:45.839 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 10 client.5200.objecter successfully canceled tick 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa12c078e00 msgr2=0x7fa12c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa12c078e00 0x7fa12c07b2c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa154006f80 tx=0x7fa1540029b0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 10 monclient: shutdown 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c0ce6c0 msgr2=0x7fa15c0c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 --2- 10.20.193.18:0/4275683393 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa15c0ce6c0 0x7fa15c0c76a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa150036000 tx=0x7fa150004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa162f44640 1 -- 10.20.193.18:0/4275683393 reap_dead start 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 shutdown_connections 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 >> 10.20.193.18:0/4275683393 conn(0x7fa15c0cc0a0 msgr2=0x7fa15c06ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.840 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 shutdown_connections 2026-02-24T18:40:45.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 -- 10.20.193.18:0/4275683393 wait complete. 2026-02-24T18:40:45.841 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.835+0000 7fa1649ce640 1 librados: shutdown 2026-02-24T18:40:45.845 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-24T18:40:45.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 10 monclient: get_monmap_and_config 2026-02-24T18:40:45.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 10 monclient: monmap: 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:45.912998+0000 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:45.912998+0000 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:45.914 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:45.915 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:45.915 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 1 Processor -- start 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 1 -- start start 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.912+0000 7f3c3b520640 10 monclient: init 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient: _add_conns ranks=[1,2,4,0,5,3] 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3c340c8fe0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3c340c93c0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:45.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f3c34159df0 0x7f3c3415a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-b con 0x7f3c34159df0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3415a7c0 con 0x7f3c340c8fe0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c34159790 con 0x7f3c340c93c0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3406e040 con 0x7f3c34159df0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3b520640 10 monclient(hunting): authenticate will time out at 1789.594185s 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3ad1f640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f3c34159df0 0x7f3c3415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c39d1d640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f3c340c93c0 0x7f3c3415c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:41794/0 (socket says 10.20.193.18:41794) 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c39d1d640 1 -- 10.20.193.18:0/1928101065 learned_addr learned my addr 10.20.193.18:0/1928101065 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3ad1f640 10 monclient(hunting): get_auth_request con 0x7f3c34159df0 auth_method 0 2026-02-24T18:40:45.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3ad1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3ad1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.913+0000 7f3c3ad1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_done global_id 5203 payload 274 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/1928101065 >> v1:10.20.193.110:6789/0 conn(0x7f3c340c93c0 legacy=0x7f3c3415c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/1928101065 >> v1:10.20.193.32:6789/0 conn(0x7f3c340c8fe0 legacy=0x7f3c3406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 1 monclient: found mon.noname-b 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.920 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.914+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/1928101065 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3406e290 con 0x7f3c34159df0 2026-02-24T18:40:45.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.916+0000 7f3c3ad1f640 1 --2- 10.20.193.18:0/1928101065 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415a280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009dc0 tx=0x7f3c3002f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949996a98c80ad77 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 1 -- 10.20.193.18:0/1928101065 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003c070 con 0x7f3c34159df0 2026-02-24T18:40:45.924 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: dump: 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.925 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3b520640 5 monclient: authenticate success, global_id 5203 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3b520640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 1 -- 10.20.193.18:0/1928101065 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c30037440 con 0x7f3c34159df0 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 1 -- 10.20.193.18:0/1928101065 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30037810 con 0x7f3c34159df0 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3951c640 10 monclient: dump: 2026-02-24T18:40:45.926 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3b520640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.917+0000 7f3c3b520640 4 set_mon_vals no callback set 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 10 monclient: shutdown 2026-02-24T18:40:45.927 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- 10.20.193.18:0/1928101065 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 msgr2=0x7f3c3415a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 --2- 10.20.193.18:0/1928101065 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415a280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009dc0 tx=0x7f3c3002f2a0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- 10.20.193.18:0/1928101065 shutdown_connections 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 --2- 10.20.193.18:0/1928101065 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415a280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- 10.20.193.18:0/1928101065 >> 10.20.193.18:0/1928101065 conn(0x7f3c340cd480 msgr2=0x7f3c340cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- 10.20.193.18:0/1928101065 shutdown_connections 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- 10.20.193.18:0/1928101065 wait complete. 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 10 monclient: build_initial_monmap 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 10 monclient: monmap: 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:45.928 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 librados: starting msgr at 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 librados: starting objecter 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 Processor -- start 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 -- start start 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 librados: setting wanted keys 2026-02-24T18:40:45.929 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 1 librados: calling monclient init 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.918+0000 7f3c3b520640 10 monclient: init 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3c340c8fe0 0x7f3c341f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-c con 0x7f3c340c8fe0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 0x7f3c3415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-a con 0x7f3c340c93c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): picked mon.noname-b con 0x7f3c34159df0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406eb30 con 0x7f3c340c93c0 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.930 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406e850 con 0x7f3c34159df0 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406ee10 con 0x7f3c340c8fe0 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c3b520640 10 monclient(hunting): authenticate will time out at 1789.600353s 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 0x7f3c3415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 0x7f3c3415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49456/0 (socket says 10.20.193.18:49456) 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 1 -- 10.20.193.18:0/4102294579 learned_addr learned my addr 10.20.193.18:0/4102294579 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 10 monclient(hunting): get_auth_request con 0x7f3c340c93c0 auth_method 0 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.919+0000 7f3c39d1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.931 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c39d1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c39d1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c39d1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): get_auth_request con 0x7f3c34159df0 auth_method 0 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3a51e640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3c340c8fe0 0x7f3c341f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:45.932 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3a51e640 10 monclient(hunting): get_auth_request con 0x7f3c340c8fe0 auth_method 0 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3a51e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3a51e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.920+0000 7f3c3a51e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3a51e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3a51e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3a51e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 10 monclient(hunting): handle_auth_done global_id 5206 payload 824 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/4102294579 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3c340c8fe0 msgr2=0x7f3c341f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3c340c8fe0 0x7f3c341f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/4102294579 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 msgr2=0x7f3c3415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 0x7f3c3415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 10 monclient: _finish_hunting 0 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 monclient: found mon.noname-b 2026-02-24T18:40:45.933 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:45.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.921+0000 7f3c3ad1f640 1 -- 10.20.193.18:0/4102294579 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340cbf90 con 0x7f3c34159df0 2026-02-24T18:40:45.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.923+0000 7f3c3ad1f640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415c790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c30036000 tx=0x7f3c30004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:45.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.923+0000 7f3c1affd640 10 client.?.objecter ms_handle_connect 0x7f3c34159df0 2026-02-24T18:40:45.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.923+0000 7f3c1affd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:45.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003c050 con 0x7f3c34159df0 2026-02-24T18:40:45.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: dump: 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.936 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: _finish_auth 0 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3003d040 con 0x7f3c34159df0 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30037e60 con 0x7f3c34159df0 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c1affd640 10 monclient: dump: 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.937 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c38d1b640 4 set_mon_vals no callback set 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 5 monclient: authenticate success, global_id 5206 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 10 monclient: _renew_subs 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c340cc270 con 0x7f3c34159df0 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 10 client.5206.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:45.938 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 10 monclient: _renew_subs 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c340cc830 con 0x7f3c34159df0 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.924+0000 7f3c3b520640 1 librados: init done 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.925+0000 7f3c3b520640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.925+0000 7f3c3b520640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.925+0000 7f3c3b520640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:45.939 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:45.925+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3415a280 con 0x7f3c34159df0 2026-02-24T18:40:46.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.028+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3c30035b00 con 0x7f3c34159df0 2026-02-24T18:40:46.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.029+0000 7f3c1affd640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 0x7f3c0407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.029+0000 7f3c3a51e640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 0x7f3c0407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.029+0000 7f3c3a51e640 10 monclient: get_auth_request con 0x7f3c04078e00 auth_method 0 2026-02-24T18:40:46.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.030+0000 7f3c1affd640 10 client.5206.objecter ms_handle_connect 0x7f3c04078e00 2026-02-24T18:40:46.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.030+0000 7f3c3a51e640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 0x7f3c0407b2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004660 tx=0x7f3c24009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.031+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f3c30084120 con 0x7f3c34159df0 2026-02-24T18:40:46.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.031+0000 7f3c1affd640 10 client.5206.objecter ms_dispatch2 0x7f3c34000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:46.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.031+0000 7f3c1affd640 3 client.5206.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:46.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.031+0000 7f3c1affd640 3 client.5206.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:46.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.031+0000 7f3c1affd640 20 client.5206.objecter dump_active .. 0 homeless 2026-02-24T18:40:46.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.038+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3c30035e20 con 0x7f3c34159df0 2026-02-24T18:40:46.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.038+0000 7f3c1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.038+0000 7f3c1affd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:46.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.122+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3c341561e0 con 0x7f3c04078e00 2026-02-24T18:40:46.124 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.122+0000 7f3c1affd640 1 -- 10.20.193.18:0/4102294579 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3c341561e0 con 0x7f3c04078e00 2026-02-24T18:40:46.124 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 5 to deep-scrub 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 10 librados: watch_flush enter 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 10 librados: watch_flush exit 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 20 client.5206.objecter shutdown clearing up homeless session... 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 10 client.5206.objecter successfully canceled tick 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 msgr2=0x7f3c0407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.123+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 0x7f3c0407b2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004660 tx=0x7f3c24009110 comp rx=0 tx=0).stop 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 10 monclient: shutdown 2026-02-24T18:40:46.126 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 msgr2=0x7f3c3415c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415c790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c30036000 tx=0x7f3c30004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 shutdown_connections 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f3c04078e00 0x7f3c0407b2c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f3c34159df0 0x7f3c3415c790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f3c340c93c0 0x7f3c3415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 --2- 10.20.193.18:0/4102294579 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f3c340c8fe0 0x7f3c341f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 >> 10.20.193.18:0/4102294579 conn(0x7f3c340cd480 msgr2=0x7f3c341513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 shutdown_connections 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 -- 10.20.193.18:0/4102294579 wait complete. 2026-02-24T18:40:46.127 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.124+0000 7f3c3b520640 1 librados: shutdown 2026-02-24T18:40:46.135 INFO:tasks.ceph:Scrubbing osd.9 2026-02-24T18:40:46.135 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:46.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 10 monclient: get_monmap_and_config 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 10 monclient: monmap: 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:46.204109+0000 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:46.204109+0000 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:46.205 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:46.206 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:46.206 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:46.206 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:46.206 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:46.206 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 1 Processor -- start 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 1 -- start start 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.203+0000 7fab5a4a9640 10 monclient: init 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient: _add_conns ranks=[3,2,1,0,5,4] 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fab541503e0 0x7fab54159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-a con 0x7fab541503e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:46.209 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fab5415a070 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fab540704c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5415a570 con 0x7fab540704c0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5415a850 con 0x7fab5415a070 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fab540c7d70 con 0x7fab541503e0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab5a4a9640 10 monclient(hunting): authenticate will time out at 1789.885291s 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab53fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fab541503e0 0x7fab54159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab58a1f640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fab540704c0 0x7fab540c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34784/0 (socket says 10.20.193.18:34784) 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab58a1f640 1 -- 10.20.193.18:0/3835422707 learned_addr learned my addr 10.20.193.18:0/3835422707 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab53fff640 10 monclient(hunting): get_auth_request con 0x7fab541503e0 auth_method 0 2026-02-24T18:40:46.210 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab53fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.204+0000 7fab53fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.205+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.205+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.205+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376339873 0 0) 0x7fab5415a570 con 0x7fab540704c0 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): my global_id is 5209 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab38003620 con 0x7fab540704c0 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 329982662 0 0) 0x7fab5415a850 con 0x7fab5415a070 2026-02-24T18:40:46.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 10 monclient(hunting): my global_id is 5192 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab5415a570 con 0x7fab5415a070 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 10 monclient(hunting): handle_auth_done global_id 5271 payload 274 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 1 -- 10.20.193.18:0/3835422707 >> v1:10.20.193.110:6789/0 conn(0x7fab5415a070 legacy=0x7fab5406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 1 -- 10.20.193.18:0/3835422707 >> v1:10.20.193.32:6789/0 conn(0x7fab540704c0 legacy=0x7fab540c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 1 monclient: found mon.noname-a 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.206+0000 7fab53fff640 1 -- 10.20.193.18:0/3835422707 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab540c7fc0 con 0x7fab541503e0 2026-02-24T18:40:46.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.207+0000 7fab53fff640 1 --2- 10.20.193.18:0/3835422707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab541503e0 0x7fab54159b30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab400098a0 tx=0x7fab4002ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284372552701382b server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.218 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4003c070 con 0x7fab541503e0 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: dump: 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.219 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4002fc10 con 0x7fab541503e0 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 5 monclient: authenticate success, global_id 5271 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 4 set_mon_vals no callback set 2026-02-24T18:40:46.220 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 1 -- 10.20.193.18:0/3835422707 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40037440 con 0x7fab541503e0 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab52ffd640 10 monclient: dump: 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.221 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 10 monclient: shutdown 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 -- 10.20.193.18:0/3835422707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab541503e0 msgr2=0x7fab54159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 --2- 10.20.193.18:0/3835422707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab541503e0 0x7fab54159b30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab400098a0 tx=0x7fab4002ef00 comp rx=0 tx=0).stop 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 -- 10.20.193.18:0/3835422707 shutdown_connections 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 --2- 10.20.193.18:0/3835422707 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab541503e0 0x7fab54159b30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 -- 10.20.193.18:0/3835422707 >> 10.20.193.18:0/3835422707 conn(0x7fab540cd600 msgr2=0x7fab540cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 -- 10.20.193.18:0/3835422707 shutdown_connections 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.209+0000 7fab5a4a9640 1 -- 10.20.193.18:0/3835422707 wait complete. 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient: monmap: 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.222 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 librados: starting msgr at 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 librados: starting objecter 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 Processor -- start 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 -- start start 2026-02-24T18:40:46.223 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 librados: setting wanted keys 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 librados: calling monclient init 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient: init 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 0x7fab541ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-a con 0x7fab540704c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fab541503e0 0x7fab5415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-b con 0x7fab541503e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fab5415a070 0x7fab5414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): picked mon.noname-c con 0x7fab5415a070 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab541507c0 con 0x7fab540704c0 2026-02-24T18:40:46.224 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab540c8610 con 0x7fab541503e0 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab54150940 con 0x7fab5415a070 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.210+0000 7fab5a4a9640 10 monclient(hunting): authenticate will time out at 1789.891773s 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 0x7fab541ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fab5415a070 0x7fab5414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fab541503e0 0x7fab5415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 0x7fab541ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49486/0 (socket says 10.20.193.18:49486) 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 1 -- 10.20.193.18:0/2169099277 learned_addr learned my addr 10.20.193.18:0/2169099277 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): get_auth_request con 0x7fab540704c0 auth_method 0 2026-02-24T18:40:46.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): get_auth_request con 0x7fab5415a070 auth_method 0 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): get_auth_request con 0x7fab541503e0 auth_method 0 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.226 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.211+0000 7fab58a1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 10 monclient(hunting): handle_auth_done global_id 5274 payload 824 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 -- 10.20.193.18:0/2169099277 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fab5415a070 msgr2=0x7fab5414cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fab5415a070 0x7fab5414cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 -- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fab541503e0 msgr2=0x7fab5415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fab541503e0 0x7fab5415c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 monclient: found mon.noname-a 2026-02-24T18:40:46.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.212+0000 7fab53fff640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5415af50 con 0x7fab540704c0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.213+0000 7fab53fff640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 0x7fab541ef620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab4003a040 tx=0x7fab400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.213+0000 7fab50ff9640 10 client.?.objecter ms_handle_connect 0x7fab540704c0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.213+0000 7fab50ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40046020 con 0x7fab540704c0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: dump: 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.228 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 5 monclient: authenticate success, global_id 5274 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab40042450 con 0x7fab540704c0 2026-02-24T18:40:46.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4003c040 con 0x7fab540704c0 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab527fc640 4 set_mon_vals no callback set 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab50ff9640 10 monclient: dump: 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.230 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 monclient: _renew_subs 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5415b230 con 0x7fab540704c0 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 client.5274.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 monclient: _renew_subs 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5415b7f0 con 0x7fab540704c0 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 1 librados: init done 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 client.5274.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 15 client.5274.objecter _session_command_op_assign -1 1 2026-02-24T18:40:46.231 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 20 client.5274.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:46.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 client.5274.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:46.232 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.215+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab54159b30 con 0x7fab540704c0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.227+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fab4004a3b0 con 0x7fab540704c0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.228+0000 7fab50ff9640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fab24078e50 0x7fab2407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.228+0000 7fab537fe640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fab24078e50 0x7fab2407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.228+0000 7fab537fe640 10 monclient: get_auth_request con 0x7fab24078e50 auth_method 0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.228+0000 7fab50ff9640 10 client.5274.objecter ms_handle_connect 0x7fab24078e50 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.228+0000 7fab537fe640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fab24078e50 0x7fab2407b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fab44003fd0 tx=0x7fab44002a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fab40082cb0 con 0x7fab540704c0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 10 client.5274.objecter ms_dispatch2 0x7fab54000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 3 client.5274.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:46.234 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 3 client.5274.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 10 client.5274.objecter checking command 1 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7fab24081330 0x7fab24083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter _get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 15 client.5274.objecter _session_command_op_remove -1 1 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter _get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 15 client.5274.objecter _session_command_op_assign 9 1 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 4 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 10 client.5274.objecter _send_command 1 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab400068f0 con 0x7fab24081330 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 20 client.5274.objecter dump_active .. 0 homeless 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab40083070 con 0x7fab540704c0 2026-02-24T18:40:46.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab50ff9640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:46.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.230+0000 7fab58a1f640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7fab24081330 0x7fab24083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.231+0000 7fab58a1f640 10 monclient: get_auth_request con 0x7fab24081330 auth_method 0 2026-02-24T18:40:46.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.232+0000 7fab50ff9640 10 client.5274.objecter ms_handle_connect 0x7fab24081330 2026-02-24T18:40:46.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.232+0000 7fab58a1f640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7fab24081330 0x7fab24083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.242+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== osd.9 v2:10.20.193.32:6816/1806399530 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fab400068f0 con 0x7fab24081330 2026-02-24T18:40:46.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.242+0000 7fab50ff9640 10 client.5274.objecter ms_dispatch2 0x7fab54000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:46.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.242+0000 7fab50ff9640 10 client.5274.objecter _finish_command 1 = system:0 2026-02-24T18:40:46.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.242+0000 7fab50ff9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 4 2026-02-24T18:40:46.244 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.242+0000 7fab50ff9640 15 client.5274.objecter _session_command_op_remove 9 1 2026-02-24T18:40:46.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 10 client.5274.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:46.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 15 client.5274.objecter _session_command_op_assign -1 2 2026-02-24T18:40:46.249 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 20 client.5274.objecter _get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 20 client.5274.objecter _get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 15 client.5274.objecter _session_command_op_remove -1 2 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 20 client.5274.objecter get_session s=0x7fab24080c60 osd=9 3 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 15 client.5274.objecter _session_command_op_assign 9 2 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 4 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 10 client.5274.objecter _send_command 2 2026-02-24T18:40:46.250 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.247+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 --> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fab541574e0 con 0x7fab24081330 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab50ff9640 1 -- 10.20.193.18:0/2169099277 <== osd.9 v2:10.20.193.32:6816/1806399530 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7fab541574e0 con 0x7fab24081330 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab50ff9640 10 client.5274.objecter ms_dispatch2 0x7fab54000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab50ff9640 10 client.5274.objecter _finish_command 2 = system:0 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab50ff9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 4 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab50ff9640 15 client.5274.objecter _session_command_op_remove 9 2 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:46.264 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:46.265 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 10 librados: watch_flush enter 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 10 librados: watch_flush exit 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 10 client.5274.objecter close_session for osd.9 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7fab24081330 msgr2=0x7fab24083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.32:6816/1806399530,v1:10.20.193.32:6817/1806399530] conn(0x7fab24081330 0x7fab24083750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 20 client.5274.objecter put_session s=0x7fab24080c60 osd=9 1 2026-02-24T18:40:46.266 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 20 client.5274.objecter shutdown clearing up homeless session... 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 10 client.5274.objecter successfully canceled tick 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fab24078e50 msgr2=0x7fab2407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fab24078e50 0x7fab2407b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fab44003fd0 tx=0x7fab44002a40 comp rx=0 tx=0).stop 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 10 monclient: shutdown 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 msgr2=0x7fab541ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 --2- 10.20.193.18:0/2169099277 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fab540704c0 0x7fab541ef620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab4003a040 tx=0x7fab400047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab58a1f640 1 -- 10.20.193.18:0/2169099277 reap_dead start 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 shutdown_connections 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 >> 10.20.193.18:0/2169099277 conn(0x7fab540cd600 msgr2=0x7fab5414f200 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 shutdown_connections 2026-02-24T18:40:46.267 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 -- 10.20.193.18:0/2169099277 wait complete. 2026-02-24T18:40:46.268 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.262+0000 7fab5a4a9640 1 librados: shutdown 2026-02-24T18:40:46.272 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.340+0000 7f6765af1640 10 monclient: get_monmap_and_config 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.340+0000 7f6765af1640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.340+0000 7f6765af1640 10 monclient: monmap: 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:46.341369+0000 2026-02-24T18:40:46.342 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:46.341369+0000 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:46.343 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.340+0000 7f6765af1640 1 Processor -- start 2026-02-24T18:40:46.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 -- start start 2026-02-24T18:40:46.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient: init 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient: _add_conns ranks=[0,4,5,1,3,2] 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f676006ff20 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f67600c99a0 0x7f6760070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-b con 0x7f67600c99a0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f676015a5e0 0x7f676014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-c con 0x7f676015a5e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67600c7e90 con 0x7f676006ff20 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6760159a40 con 0x7f67600c99a0 2026-02-24T18:40:46.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f67600c8170 con 0x7f676015a5e0 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f6765af1640 10 monclient(hunting): authenticate will time out at 1790.022570s 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f675f7fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f676006ff20 0x7f6760159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48650/0 (socket says 10.20.193.18:48650) 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.341+0000 7f675f7fe640 1 -- 10.20.193.18:0/3636585720 learned_addr learned my addr 10.20.193.18:0/3636585720 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 1 --2- 10.20.193.18:0/3636585720 >> v2:10.20.193.32:3300/0 conn(0x7f67600c99a0 0x7f6760070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 1 --2- 10.20.193.18:0/3636585720 >> v2:10.20.193.110:3300/0 conn(0x7f676015a5e0 0x7f676014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): get_auth_request con 0x7f676015a5e0 auth_method 0 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): get_auth_request con 0x7f67600c99a0 auth_method 0 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.342+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 1 -- 10.20.193.18:0/3636585720 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826519798 0 0) 0x7f67600c7e90 con 0x7f676006ff20 2026-02-24T18:40:46.349 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 10 monclient(hunting): _negotiate 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 10 monclient(hunting): my global_id is 5277 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675e7fc640 1 -- 10.20.193.18:0/3636585720 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6744003620 con 0x7f676006ff20 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 10 monclient(hunting): handle_auth_done global_id 5198 payload 274 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 1 -- 10.20.193.18:0/3636585720 >> v2:10.20.193.32:3300/0 conn(0x7f67600c99a0 msgr2=0x7f6760070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 1 --2- 10.20.193.18:0/3636585720 >> v2:10.20.193.32:3300/0 conn(0x7f67600c99a0 0x7f6760070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 1 -- 10.20.193.18:0/3636585720 >> v1:10.20.193.18:6789/0 conn(0x7f676006ff20 legacy=0x7f6760159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.350 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.343+0000 7f675ffff640 1 -- 10.20.193.18:0/3636585720 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67600c8450 con 0x7f676015a5e0 2026-02-24T18:40:46.351 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.344+0000 7f675ffff640 1 --2- 10.20.193.18:0/3636585720 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f676015a5e0 0x7f676014fe10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6754009d90 tx=0x7f675402f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94bb20f1bff5c00f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 1 -- 10.20.193.18:0/3636585720 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675403c070 con 0x7f676015a5e0 2026-02-24T18:40:46.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: dump: 2026-02-24T18:40:46.357 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.358 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 1 -- 10.20.193.18:0/3636585720 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6754037440 con 0x7f676015a5e0 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 5 monclient: authenticate success, global_id 5198 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 1 -- 10.20.193.18:0/3636585720 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754037810 con 0x7f676015a5e0 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 4 set_mon_vals no callback set 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f675e7fc640 10 monclient: dump: 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.359 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 10 monclient: shutdown 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 -- 10.20.193.18:0/3636585720 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f676015a5e0 msgr2=0x7f676014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 --2- 10.20.193.18:0/3636585720 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f676015a5e0 0x7f676014fe10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6754009d90 tx=0x7f675402f1e0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.360 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 -- 10.20.193.18:0/3636585720 shutdown_connections 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 --2- 10.20.193.18:0/3636585720 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f676015a5e0 0x7f676014fe10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 --2- 10.20.193.18:0/3636585720 >> v2:10.20.193.32:3300/0 conn(0x7f67600c99a0 0x7f6760070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 -- 10.20.193.18:0/3636585720 >> 10.20.193.18:0/3636585720 conn(0x7f67600cd780 msgr2=0x7f67600cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 -- 10.20.193.18:0/3636585720 shutdown_connections 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.346+0000 7f6765af1640 1 -- 10.20.193.18:0/3636585720 wait complete. 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient: monmap: 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.361 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 librados: starting msgr at 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 librados: starting objecter 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 Processor -- start 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 -- start start 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 librados: setting wanted keys 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 librados: calling monclient init 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient: init 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.362 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 0x7f676006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-b con 0x7f676006ff20 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-c con 0x7f67600c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f676015a5e0 0x7f67600ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): picked mon.noname-a con 0x7f676015a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6760150350 con 0x7f676015a5e0 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67600c8a10 con 0x7f676006ff20 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67601504d0 con 0x7f67600c99a0 2026-02-24T18:40:46.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.347+0000 7f6765af1640 10 monclient(hunting): authenticate will time out at 1790.028963s 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 0x7f676006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f676015a5e0 0x7f67600ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 0x7f676006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52282/0 (socket says 10.20.193.18:52282) 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 1 -- 10.20.193.18:0/1991949316 learned_addr learned my addr 10.20.193.18:0/1991949316 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37922/0 (socket says 10.20.193.18:37922) 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): get_auth_request con 0x7f676006ff20 auth_method 0 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): get_auth_request con 0x7f67600c99a0 auth_method 0 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): get_auth_request con 0x7f676015a5e0 auth_method 0 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.365 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.348+0000 7f675ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 10 monclient(hunting): handle_auth_done global_id 5201 payload 824 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 -- 10.20.193.18:0/1991949316 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 msgr2=0x7f676006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 0x7f676006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 -- 10.20.193.18:0/1991949316 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f676015a5e0 msgr2=0x7f67600ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f676015a5e0 0x7f67600ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.349+0000 7f675effd640 1 -- 10.20.193.18:0/1991949316 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676006f3d0 con 0x7f67600c99a0 2026-02-24T18:40:46.366 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.350+0000 7f675effd640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9b0 tx=0x7f675000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.350+0000 7f673bfff640 10 client.?.objecter ms_handle_connect 0x7f67600c99a0 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.350+0000 7f673bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750017070 con 0x7f67600c99a0 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: dump: 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.367 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 5 monclient: authenticate success, global_id 5201 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _renew_subs 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676006d4d0 con 0x7f67600c99a0 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 client.5201.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _renew_subs 2026-02-24T18:40:46.368 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676006da90 con 0x7f67600c99a0 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 1 librados: init done 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675000ca00 con 0x7f67600c99a0 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750021060 con 0x7f67600c99a0 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f673bfff640 10 monclient: dump: 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.369 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f675dffb640 4 set_mon_vals no callback set 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.352+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67600c62a0 con 0x7f67600c99a0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.364+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6750026480 con 0x7f67600c99a0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.364+0000 7f673bfff640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 0x7f673007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.364+0000 7f675f7fe640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 0x7f673007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.364+0000 7f675f7fe640 10 monclient: get_auth_request con 0x7f6730078e30 auth_method 0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.365+0000 7f675f7fe640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 0x7f673007b2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f674c005980 tx=0x7f674c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.365+0000 7f673bfff640 10 client.5201.objecter ms_handle_connect 0x7f6730078e30 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.366+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f675000cdb0 con 0x7f67600c99a0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.366+0000 7f673bfff640 10 client.5201.objecter ms_dispatch2 0x7f6760000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.366+0000 7f673bfff640 3 client.5201.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:46.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.366+0000 7f673bfff640 3 client.5201.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:46.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.366+0000 7f673bfff640 20 client.5201.objecter dump_active .. 0 homeless 2026-02-24T18:40:46.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.467+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f67600c62a0 con 0x7f67600c99a0 2026-02-24T18:40:46.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.467+0000 7f673bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.467+0000 7f673bfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:46.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.549+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "9", "target": ["mon-mgr", ""]}) -- 0x7f676006d280 con 0x7f6730078e30 2026-02-24T18:40:46.551 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.549+0000 7f673bfff640 1 -- 10.20.193.18:0/1991949316 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 9 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f676006d280 con 0x7f6730078e30 2026-02-24T18:40:46.551 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 9 to deep-scrub 2026-02-24T18:40:46.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 10 librados: watch_flush enter 2026-02-24T18:40:46.552 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 10 librados: watch_flush exit 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 20 client.5201.objecter shutdown clearing up homeless session... 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 10 client.5201.objecter successfully canceled tick 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 msgr2=0x7f673007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 0x7f673007b2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f674c005980 tx=0x7f674c009040 comp rx=0 tx=0).stop 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 10 monclient: shutdown 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 msgr2=0x7f67601f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9b0 tx=0x7f675000ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 shutdown_connections 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f6730078e30 0x7f673007b2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f676015a5e0 0x7f67600ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.553 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f67600c99a0 0x7f67601f3eb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 --2- 10.20.193.18:0/1991949316 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f676006ff20 0x7f676006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 >> 10.20.193.18:0/1991949316 conn(0x7f67600cd780 msgr2=0x7f676014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 shutdown_connections 2026-02-24T18:40:46.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 -- 10.20.193.18:0/1991949316 wait complete. 2026-02-24T18:40:46.554 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.550+0000 7f6765af1640 1 librados: shutdown 2026-02-24T18:40:46.560 INFO:tasks.ceph:Scrubbing osd.13 2026-02-24T18:40:46.560 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:46.630 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 10 monclient: get_monmap_and_config 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 10 monclient: monmap: 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:46.630272+0000 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:46.630272+0000 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.631 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:46.632 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 1 Processor -- start 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 1 -- start start 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.629+0000 7fa3800f3640 10 monclient: init 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient: _add_conns ranks=[3,0,5,4,1,2] 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa378155ca0 0x7fa378073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-a con 0x7fa378155ca0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa37815cb70 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:46.635 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fa378070430 0x7fa37815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-c con 0x7fa378070430 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa378073d40 con 0x7fa37815cb70 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fa378072510 con 0x7fa378155ca0 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa378072760 con 0x7fa378070430 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa3800f3640 10 monclient(hunting): authenticate will time out at 1790.311439s 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.630+0000 7fa37de68640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fa378155ca0 0x7fa378073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37d667640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fa37815cb70 0x7fa37806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48652/0 (socket says 10.20.193.18:48652) 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37d667640 1 -- 10.20.193.18:0/2210292533 learned_addr learned my addr 10.20.193.18:0/2210292533 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:46.636 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 1 --2- 10.20.193.18:0/2210292533 >> v2:10.20.193.110:3300/0 conn(0x7fa378070430 0x7fa37815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): get_auth_request con 0x7fa378155ca0 auth_method 0 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): get_auth_request con 0x7fa378070430 auth_method 0 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.637 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.631+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 10 monclient(hunting): handle_auth_done global_id 5204 payload 274 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 1 -- 10.20.193.18:0/2210292533 >> v2:10.20.193.18:3300/0 conn(0x7fa378155ca0 msgr2=0x7fa378073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 1 --2- 10.20.193.18:0/2210292533 >> v2:10.20.193.18:3300/0 conn(0x7fa378155ca0 0x7fa378073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 1 -- 10.20.193.18:0/2210292533 >> v1:10.20.193.18:6789/0 conn(0x7fa37815cb70 legacy=0x7fa37806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.632+0000 7fa37e669640 1 -- 10.20.193.18:0/2210292533 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa378072a40 con 0x7fa378070430 2026-02-24T18:40:46.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.633+0000 7fa37e669640 1 --2- 10.20.193.18:0/2210292533 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815f450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa378074fd0 tx=0x7fa37402f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5487f6ec02e0e5a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 1 -- 10.20.193.18:0/2210292533 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37403c070 con 0x7fa378070430 2026-02-24T18:40:46.644 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: dump: 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.645 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 1 -- 10.20.193.18:0/2210292533 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37402fe40 con 0x7fa378070430 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 1 -- 10.20.193.18:0/2210292533 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374037660 con 0x7fa378070430 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 5 monclient: authenticate success, global_id 5204 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 4 set_mon_vals no callback set 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa37ce66640 10 monclient: dump: 2026-02-24T18:40:46.646 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 10 monclient: shutdown 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.647 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 -- 10.20.193.18:0/2210292533 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 msgr2=0x7fa37815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 --2- 10.20.193.18:0/2210292533 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815f450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa378074fd0 tx=0x7fa37402f130 comp rx=0 tx=0).stop 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 -- 10.20.193.18:0/2210292533 shutdown_connections 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 --2- 10.20.193.18:0/2210292533 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815f450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 --2- 10.20.193.18:0/2210292533 >> v2:10.20.193.18:3300/0 conn(0x7fa378155ca0 0x7fa378073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 -- 10.20.193.18:0/2210292533 >> 10.20.193.18:0/2210292533 conn(0x7fa3780cd600 msgr2=0x7fa3780cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 -- 10.20.193.18:0/2210292533 shutdown_connections 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.635+0000 7fa3800f3640 1 -- 10.20.193.18:0/2210292533 wait complete. 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient: monmap: 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:46.648 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 librados: starting msgr at 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 librados: starting objecter 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 Processor -- start 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 -- start start 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 librados: setting wanted keys 2026-02-24T18:40:46.649 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 librados: calling monclient init 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient: init 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-c con 0x7fa378070430 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa378155ca0 0x7fa378149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-a con 0x7fa378155ca0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa37815cb70 0x7fa378153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): picked mon.noname-b con 0x7fa37815cb70 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:46.650 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780732e0 con 0x7fa378155ca0 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa378073000 con 0x7fa37815cb70 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa378156080 con 0x7fa378070430 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.636+0000 7fa3800f3640 10 monclient(hunting): authenticate will time out at 1790.317943s 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa378155ca0 0x7fa378149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa37815cb70 0x7fa378153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa378155ca0 0x7fa378149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49518/0 (socket says 10.20.193.18:49518) 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 1 -- 10.20.193.18:0/544395392 learned_addr learned my addr 10.20.193.18:0/544395392 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:46.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): get_auth_request con 0x7fa378155ca0 auth_method 0 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): get_auth_request con 0x7fa378070430 auth_method 0 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): get_auth_request con 0x7fa37815cb70 auth_method 0 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37d667640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37de68640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.637+0000 7fa37e669640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 10 monclient(hunting): handle_auth_done global_id 5207 payload 824 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 -- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa37815cb70 msgr2=0x7fa378153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa37815cb70 0x7fa378153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 -- 10.20.193.18:0/544395392 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa378155ca0 msgr2=0x7fa378149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa378155ca0 0x7fa378149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.639+0000 7fa37de68640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3781f7300 con 0x7fa378070430 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.640+0000 7fa37de68640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815b310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9e0 tx=0x7fa36800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.640+0000 7fa3667fc640 10 client.?.objecter ms_handle_connect 0x7fa378070430 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.640+0000 7fa3667fc640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368017070 con 0x7fa378070430 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 10 monclient: dump: 2026-02-24T18:40:46.654 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 5 monclient: authenticate success, global_id 5207 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 monclient: _renew_subs 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa378159cd0 con 0x7fa378070430 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 client.5207.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 monclient: _renew_subs 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37815a270 con 0x7fa378070430 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 1 librados: init done 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 client.5207.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 15 client.5207.objecter _session_command_op_assign -1 1 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 20 client.5207.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 client.5207.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.641+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa37806d4a0 con 0x7fa378070430 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37806d4a0 con 0x7fa378070430 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368021060 con 0x7fa378070430 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa3667fc640 10 monclient: dump: 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.657 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.642+0000 7fa367fff640 4 set_mon_vals no callback set 2026-02-24T18:40:46.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.654+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa368024060 con 0x7fa378070430 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.654+0000 7fa3667fc640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa348078e00 0x7fa34807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa37d667640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa348078e00 0x7fa34807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa37d667640 10 monclient: get_auth_request con 0x7fa348078e00 auth_method 0 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 10 client.5207.objecter ms_handle_connect 0x7fa348078e00 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa37d667640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa348078e00 0x7fa34807b2c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa37814b0c0 tx=0x7fa36c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa368097410 con 0x7fa378070430 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 10 client.5207.objecter ms_dispatch2 0x7fa378000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 3 client.5207.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 3 client.5207.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:46.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 10 client.5207.objecter checking command 1 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7fa3480812e0 0x7fa348083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter _get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 15 client.5207.objecter _session_command_op_remove -1 1 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter _get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 15 client.5207.objecter _session_command_op_assign 13 1 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 4 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 10 client.5207.objecter _send_command 1 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa36800be50 con 0x7fa3480812e0 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 20 client.5207.objecter dump_active .. 0 homeless 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa36805ded0 con 0x7fa378070430 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.655+0000 7fa3667fc640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.656+0000 7fa37e669640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7fa3480812e0 0x7fa348083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.656+0000 7fa37e669640 10 monclient: get_auth_request con 0x7fa3480812e0 auth_method 0 2026-02-24T18:40:46.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.658+0000 7fa3667fc640 10 client.5207.objecter ms_handle_connect 0x7fa3480812e0 2026-02-24T18:40:46.663 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.658+0000 7fa37e669640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7fa3480812e0 0x7fa348083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.667+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== osd.13 v2:10.20.193.32:6824/316721302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa36800be50 con 0x7fa3480812e0 2026-02-24T18:40:46.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.667+0000 7fa3667fc640 10 client.5207.objecter ms_dispatch2 0x7fa378000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:46.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.667+0000 7fa3667fc640 10 client.5207.objecter _finish_command 1 = system:0 2026-02-24T18:40:46.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.667+0000 7fa3667fc640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 4 2026-02-24T18:40:46.669 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.667+0000 7fa3667fc640 15 client.5207.objecter _session_command_op_remove 13 1 2026-02-24T18:40:46.675 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 10 client.5207.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 15 client.5207.objecter _session_command_op_assign -1 2 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 20 client.5207.objecter _get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 20 client.5207.objecter _get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 15 client.5207.objecter _session_command_op_remove -1 2 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 20 client.5207.objecter get_session s=0x7fa348080c10 osd=13 3 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 15 client.5207.objecter _session_command_op_assign 13 2 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 4 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 10 client.5207.objecter _send_command 2 2026-02-24T18:40:46.676 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.674+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 --> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa37806efe0 con 0x7fa3480812e0 2026-02-24T18:40:46.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.680+0000 7fa3667fc640 1 -- 10.20.193.18:0/544395392 <== osd.13 v2:10.20.193.32:6824/316721302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa37806efe0 con 0x7fa3480812e0 2026-02-24T18:40:46.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.680+0000 7fa3667fc640 10 client.5207.objecter ms_dispatch2 0x7fa378000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:46.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.680+0000 7fa3667fc640 10 client.5207.objecter _finish_command 2 = system:0 2026-02-24T18:40:46.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.680+0000 7fa3667fc640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 4 2026-02-24T18:40:46.682 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.680+0000 7fa3667fc640 15 client.5207.objecter _session_command_op_remove 13 2 2026-02-24T18:40:46.683 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:46.683 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:46.683 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:46.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 10 librados: watch_flush enter 2026-02-24T18:40:46.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 10 librados: watch_flush exit 2026-02-24T18:40:46.684 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 10 client.5207.objecter close_session for osd.13 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7fa3480812e0 msgr2=0x7fa348083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.32:6824/316721302,v1:10.20.193.32:6825/316721302] conn(0x7fa3480812e0 0x7fa348083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 20 client.5207.objecter put_session s=0x7fa348080c10 osd=13 1 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 20 client.5207.objecter shutdown clearing up homeless session... 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 10 client.5207.objecter successfully canceled tick 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa348078e00 msgr2=0x7fa34807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa348078e00 0x7fa34807b2c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa37814b0c0 tx=0x7fa36c009040 comp rx=0 tx=0).stop 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 10 monclient: shutdown 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 msgr2=0x7fa37815b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 --2- 10.20.193.18:0/544395392 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa378070430 0x7fa37815b310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9e0 tx=0x7fa36800eeb0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.685 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa37e669640 1 -- 10.20.193.18:0/544395392 reap_dead start 2026-02-24T18:40:46.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 shutdown_connections 2026-02-24T18:40:46.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 >> 10.20.193.18:0/544395392 conn(0x7fa3780cd600 msgr2=0x7fa3780cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 shutdown_connections 2026-02-24T18:40:46.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 -- 10.20.193.18:0/544395392 wait complete. 2026-02-24T18:40:46.686 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.681+0000 7fa3800f3640 1 librados: shutdown 2026-02-24T18:40:46.691 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.758+0000 7fd7d43d3640 10 monclient: get_monmap_and_config 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.758+0000 7fd7d43d3640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.758+0000 7fd7d43d3640 10 monclient: monmap: 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:46.759429+0000 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:46.759429+0000 2026-02-24T18:40:46.760 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:46.761 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.758+0000 7fd7d43d3640 1 Processor -- start 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 1 -- start start 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient: init 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient: _add_conns ranks=[5,2,0,3,4,1] 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd7cc1527d0 0x7fd7cc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-c con 0x7fd7cc1527d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7cc152bb0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7cc0c6190 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7cc06e300 con 0x7fd7cc0c6190 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7cc06e5e0 con 0x7fd7cc152bb0 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd7cc06f1f0 con 0x7fd7cc1527d0 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.759+0000 7fd7d43d3640 10 monclient(hunting): authenticate will time out at 1790.440578s 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2949640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fd7cc0c6190 0x7fd7cc0c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48668/0 (socket says 10.20.193.18:48668) 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2949640 1 -- 10.20.193.18:0/2464999220 learned_addr learned my addr 10.20.193.18:0/2464999220 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 1 --2- 10.20.193.18:0/2464999220 >> v2:10.20.193.110:3300/0 conn(0x7fd7cc1527d0 0x7fd7cc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): get_auth_request con 0x7fd7cc1527d0 auth_method 0 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.760+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 1 -- 10.20.193.18:0/2464999220 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203652114 0 0) 0x7fd7cc06e300 con 0x7fd7cc0c6190 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 10 monclient(hunting): _negotiate 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 10 monclient(hunting): my global_id is 5292 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d1146640 1 -- 10.20.193.18:0/2464999220 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b4003620 con 0x7fd7cc0c6190 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 10 monclient(hunting): handle_auth_done global_id 5210 payload 274 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 1 -- 10.20.193.18:0/2464999220 >> v1:10.20.193.110:6789/0 conn(0x7fd7cc152bb0 legacy=0x7fd7cc0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 1 -- 10.20.193.18:0/2464999220 >> v1:10.20.193.18:6789/0 conn(0x7fd7cc0c6190 legacy=0x7fd7cc0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.761+0000 7fd7d2148640 1 -- 10.20.193.18:0/2464999220 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc06e8c0 con 0x7fd7cc1527d0 2026-02-24T18:40:46.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.763+0000 7fd7d2148640 1 --2- 10.20.193.18:0/2464999220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc06da60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc06f370 tx=0x7fd7bc02ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b03e230cbded287 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 1 -- 10.20.193.18:0/2464999220 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc03c070 con 0x7fd7cc1527d0 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: dump: 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.772 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 5 monclient: authenticate success, global_id 5210 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 1 -- 10.20.193.18:0/2464999220 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc02fad0 con 0x7fd7cc1527d0 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 1 -- 10.20.193.18:0/2464999220 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc037440 con 0x7fd7cc1527d0 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d1146640 10 monclient: dump: 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:46.774 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 4 set_mon_vals no callback set 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 10 monclient: shutdown 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 -- 10.20.193.18:0/2464999220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 msgr2=0x7fd7cc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/2464999220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc06da60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc06f370 tx=0x7fd7bc02ee10 comp rx=0 tx=0).stop 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 -- 10.20.193.18:0/2464999220 shutdown_connections 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/2464999220 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc06da60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 -- 10.20.193.18:0/2464999220 >> 10.20.193.18:0/2464999220 conn(0x7fd7cc0cd780 msgr2=0x7fd7cc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 -- 10.20.193.18:0/2464999220 shutdown_connections 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.764+0000 7fd7d43d3640 1 -- 10.20.193.18:0/2464999220 wait complete. 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 10 monclient: build_initial_monmap 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 10 monclient: monmap: 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:46.775 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 librados: starting msgr at 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 librados: starting objecter 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 Processor -- start 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 -- start start 2026-02-24T18:40:46.776 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 librados: setting wanted keys 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 1 librados: calling monclient init 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 10 monclient: init 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.765+0000 7fd7d43d3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 0x7fd7cc1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-a con 0x7fd7cc0c6190 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-c con 0x7fd7cc1527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd7cc152bb0 0x7fd7cc157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): picked mon.noname-b con 0x7fd7cc152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc06dfa0 con 0x7fd7cc0c6190 2026-02-24T18:40:46.777 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc06ee80 con 0x7fd7cc152bb0 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0c9570 con 0x7fd7cc1527d0 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d43d3640 10 monclient(hunting): authenticate will time out at 1790.447036s 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 0x7fd7cc1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 0x7fd7cc1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49528/0 (socket says 10.20.193.18:49528) 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 1 -- 10.20.193.18:0/1282085270 learned_addr learned my addr 10.20.193.18:0/1282085270 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): get_auth_request con 0x7fd7cc0c6190 auth_method 0 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2949640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd7cc152bb0 0x7fd7cc157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): get_auth_request con 0x7fd7cc1527d0 auth_method 0 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d1947640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2949640 10 monclient(hunting): get_auth_request con 0x7fd7cc152bb0 auth_method 0 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2949640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2949640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:46.779 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.766+0000 7fd7d2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d2949640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d2949640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d2949640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 10 monclient(hunting): handle_auth_done global_id 5213 payload 824 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 -- 10.20.193.18:0/1282085270 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd7cc152bb0 msgr2=0x7fd7cc157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd7cc152bb0 0x7fd7cc157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 -- 10.20.193.18:0/1282085270 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 msgr2=0x7fd7cc1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 0x7fd7cc1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 10 monclient: _finish_hunting 0 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 monclient: found mon.noname-c 2026-02-24T18:40:46.780 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:46.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.767+0000 7fd7d1947640 1 -- 10.20.193.18:0/1282085270 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc154b70 con 0x7fd7cc1527d0 2026-02-24T18:40:46.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.769+0000 7fd7d1947640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc1f5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000b410 tx=0x7fd7c000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.769+0000 7fd7b2ffd640 10 client.?.objecter ms_handle_connect 0x7fd7cc1527d0 2026-02-24T18:40:46.781 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.769+0000 7fd7b2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0016070 con 0x7fd7cc1527d0 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: dump: 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.783 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c00044a0 con 0x7fd7cc1527d0 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0002920 con 0x7fd7cc1527d0 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:46.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7b2ffd640 10 monclient: dump: 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7d0945640 4 set_mon_vals no callback set 2026-02-24T18:40:46.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7d43d3640 5 monclient: authenticate success, global_id 5213 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7d43d3640 10 monclient: _renew_subs 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7d43d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.770+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc154e50 con 0x7fd7cc1527d0 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 client.5213.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 monclient: _renew_subs 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc155410 con 0x7fd7cc1527d0 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 1 librados: init done 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:46.786 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.771+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc06d550 con 0x7fd7cc1527d0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.782+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd7c0002ac0 con 0x7fd7cc1527d0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.783+0000 7fd7b2ffd640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 0x7fd79c07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.783+0000 7fd7d2148640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 0x7fd79c07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.783+0000 7fd7d2148640 10 monclient: get_auth_request con 0x7fd79c078e30 auth_method 0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7d2148640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 0x7fd79c07b2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009130 tx=0x7fd7bc033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 10 client.5213.objecter ms_handle_connect 0x7fd79c078e30 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd7c005d390 con 0x7fd7cc1527d0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 10 client.5213.objecter ms_dispatch2 0x7fd7cc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 3 client.5213.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 3 client.5213.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:46.788 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.784+0000 7fd7b2ffd640 20 client.5213.objecter dump_active .. 0 homeless 2026-02-24T18:40:46.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.884+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd7cc06d550 con 0x7fd7cc1527d0 2026-02-24T18:40:46.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.884+0000 7fd7b2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:46.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.884+0000 7fd7b2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:46.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.967+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "13", "target": ["mon-mgr", ""]}) -- 0x7fd7cc1562f0 con 0x7fd79c078e30 2026-02-24T18:40:46.969 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.967+0000 7fd7b2ffd640 1 -- 10.20.193.18:0/1282085270 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 13 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fd7cc1562f0 con 0x7fd79c078e30 2026-02-24T18:40:46.969 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 13 to deep-scrub 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 10 librados: watch_flush enter 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 10 librados: watch_flush exit 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 20 client.5213.objecter shutdown clearing up homeless session... 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 10 client.5213.objecter successfully canceled tick 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 msgr2=0x7fd79c07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 0x7fd79c07b2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009130 tx=0x7fd7bc033000 comp rx=0 tx=0).stop 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 10 monclient: shutdown 2026-02-24T18:40:46.971 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 msgr2=0x7fd7cc1f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc1f5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000b410 tx=0x7fd7c000b8e0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 shutdown_connections 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd79c078e30 0x7fd79c07b2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd7cc152bb0 0x7fd7cc157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd7cc1527d0 0x7fd7cc1f5160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 --2- 10.20.193.18:0/1282085270 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd7cc0c6190 0x7fd7cc1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 >> 10.20.193.18:0/1282085270 conn(0x7fd7cc0cd780 msgr2=0x7fd7cc1501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 shutdown_connections 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 -- 10.20.193.18:0/1282085270 wait complete. 2026-02-24T18:40:46.972 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:46.968+0000 7fd7d43d3640 1 librados: shutdown 2026-02-24T18:40:46.978 INFO:tasks.ceph:Scrubbing osd.2 2026-02-24T18:40:46.979 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.046+0000 7fd10dadb640 10 monclient: get_monmap_and_config 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.046+0000 7fd10dadb640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.046+0000 7fd10dadb640 10 monclient: monmap: 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:47.047518+0000 2026-02-24T18:40:47.048 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:47.047518+0000 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:47.049 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 Processor -- start 2026-02-24T18:40:47.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 -- start start 2026-02-24T18:40:47.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient: init 2026-02-24T18:40:47.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient: _add_conns ranks=[3,0,1,5,4,2] 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd1081503e0 0x7fd108159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-a con 0x7fd1081503e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd10815a070 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd1080704c0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd10815a570 con 0x7fd10815a070 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd10815a850 con 0x7fd1080704c0 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fd1080c7d70 con 0x7fd1081503e0 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.047+0000 7fd10dadb640 10 monclient(hunting): authenticate will time out at 1790.728684s 2026-02-24T18:40:47.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fd1081503e0 0x7fd108159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd107fff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7fd1080704c0 0x7fd1080c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34786/0 (socket says 10.20.193.18:34786) 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd107fff640 1 -- 10.20.193.18:0/2336332450 learned_addr learned my addr 10.20.193.18:0/2336332450 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): get_auth_request con 0x7fd1081503e0 auth_method 0 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.048+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 1 -- 10.20.193.18:0/2336332450 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603862785 0 0) 0x7fd10815a850 con 0x7fd1080704c0 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 10 monclient(hunting): _negotiate 2026-02-24T18:40:47.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 10 monclient(hunting): my global_id is 5227 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1067fc640 1 -- 10.20.193.18:0/2336332450 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ec003620 con 0x7fd1080704c0 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 10 monclient(hunting): handle_auth_done global_id 5298 payload 274 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 1 -- 10.20.193.18:0/2336332450 >> v1:10.20.193.32:6789/0 conn(0x7fd1080704c0 legacy=0x7fd1080c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 1 -- 10.20.193.18:0/2336332450 >> v1:10.20.193.18:6789/0 conn(0x7fd10815a070 legacy=0x7fd10806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.055 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.049+0000 7fd1077fe640 1 -- 10.20.193.18:0/2336332450 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1080c7fc0 con 0x7fd1081503e0 2026-02-24T18:40:47.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.051+0000 7fd1077fe640 1 --2- 10.20.193.18:0/2336332450 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1081503e0 0x7fd108159b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098a0 tx=0x7fd0f402ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8453e781327384 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 1 -- 10.20.193.18:0/2336332450 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f403c070 con 0x7fd1081503e0 2026-02-24T18:40:47.060 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: dump: 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.061 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 1 -- 10.20.193.18:0/2336332450 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f402fc10 con 0x7fd1081503e0 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd10dadb640 5 monclient: authenticate success, global_id 5298 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd10dadb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 1 -- 10.20.193.18:0/2336332450 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4037440 con 0x7fd1081503e0 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.053+0000 7fd1067fc640 10 monclient: dump: 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.062 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 4 set_mon_vals no callback set 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 10 monclient: shutdown 2026-02-24T18:40:47.063 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 -- 10.20.193.18:0/2336332450 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1081503e0 msgr2=0x7fd108159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 --2- 10.20.193.18:0/2336332450 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1081503e0 0x7fd108159b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098a0 tx=0x7fd0f402ef00 comp rx=0 tx=0).stop 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 -- 10.20.193.18:0/2336332450 shutdown_connections 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 --2- 10.20.193.18:0/2336332450 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1081503e0 0x7fd108159b30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 -- 10.20.193.18:0/2336332450 >> 10.20.193.18:0/2336332450 conn(0x7fd1080cd600 msgr2=0x7fd1080cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 -- 10.20.193.18:0/2336332450 shutdown_connections 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 1 -- 10.20.193.18:0/2336332450 wait complete. 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.054+0000 7fd10dadb640 10 monclient: monmap: 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:47.064 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 librados: starting msgr at 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 librados: starting objecter 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 Processor -- start 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 -- start start 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 librados: setting wanted keys 2026-02-24T18:40:47.065 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 librados: calling monclient init 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient: init 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 0x7fd1081f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-a con 0x7fd1080704c0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1081503e0 0x7fd1080ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-c con 0x7fd1081503e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd10815a070 0x7fd1080bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): picked mon.noname-b con 0x7fd10815a070 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081507c0 con 0x7fd1080704c0 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.066 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1080c8610 con 0x7fd10815a070 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd108150940 con 0x7fd1081503e0 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd10dadb640 10 monclient(hunting): authenticate will time out at 1790.736559s 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 0x7fd1081f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd107fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd10815a070 0x7fd1080bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 0x7fd1081f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49542/0 (socket says 10.20.193.18:49542) 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 1 -- 10.20.193.18:0/221898258 learned_addr learned my addr 10.20.193.18:0/221898258 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 10 monclient(hunting): get_auth_request con 0x7fd1080704c0 auth_method 0 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.067 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.055+0000 7fd1077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1081503e0 0x7fd1080ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): get_auth_request con 0x7fd10815a070 auth_method 0 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): get_auth_request con 0x7fd1081503e0 auth_method 0 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd1077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.068 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd107fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.056+0000 7fd106ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 10 monclient(hunting): handle_auth_done global_id 5304 payload 824 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 -- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1081503e0 msgr2=0x7fd1080ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1081503e0 0x7fd1080ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 -- 10.20.193.18:0/221898258 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd10815a070 msgr2=0x7fd1080bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd10815a070 0x7fd1080bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.069 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.057+0000 7fd1077fe640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1081f28e0 con 0x7fd1080704c0 2026-02-24T18:40:47.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.058+0000 7fd1077fe640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 0x7fd1081f19e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402f430 tx=0x7fd0f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.058+0000 7fd0e3fff640 10 client.?.objecter ms_handle_connect 0x7fd1080704c0 2026-02-24T18:40:47.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.058+0000 7fd0e3fff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f403c040 con 0x7fd1080704c0 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 10 monclient: dump: 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.072 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd0e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.073 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 5 monclient: authenticate success, global_id 5304 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 10 monclient: _renew_subs 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1080beb20 con 0x7fd1080704c0 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 10 client.5304.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 10 monclient: _renew_subs 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081f2600 con 0x7fd1080704c0 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.059+0000 7fd10dadb640 1 librados: init done 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 10 client.5304.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 15 client.5304.objecter _session_command_op_assign -1 1 2026-02-24T18:40:47.074 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 20 client.5304.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 10 client.5304.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd108159b30 con 0x7fd1080704c0 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd108159b30 con 0x7fd1080704c0 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4042610 con 0x7fd1080704c0 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd105ffb640 4 set_mon_vals no callback set 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.060+0000 7fd0e3fff640 10 monclient: dump: 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.075 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.076 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.072+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd0f40427b0 con 0x7fd1080704c0 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.073+0000 7fd0e3fff640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0d8078e30 0x7fd0d807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.073+0000 7fd106ffd640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0d8078e30 0x7fd0d807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.073+0000 7fd106ffd640 10 monclient: get_auth_request con 0x7fd0d8078e30 auth_method 0 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.073+0000 7fd106ffd640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0d8078e30 0x7fd0d807b2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1081540a0 tx=0x7fd0f8007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.073+0000 7fd0e3fff640 10 client.5304.objecter ms_handle_connect 0x7fd0d8078e30 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd0f40bc040 con 0x7fd1080704c0 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 10 client.5304.objecter ms_dispatch2 0x7fd108000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:47.078 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 3 client.5304.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 3 client.5304.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 10 client.5304.objecter checking command 1 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7fd0d8081310 0x7fd0d8083730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter _get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 15 client.5304.objecter _session_command_op_remove -1 1 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter _get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 15 client.5304.objecter _session_command_op_assign 2 1 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 4 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 10 client.5304.objecter _send_command 1 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0f403fea0 con 0x7fd0d8081310 2026-02-24T18:40:47.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 20 client.5304.objecter dump_active .. 0 homeless 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f4082d20 con 0x7fd1080704c0 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.075+0000 7fd0e3fff640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.076+0000 7fd107fff640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7fd0d8081310 0x7fd0d8083730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.076+0000 7fd107fff640 10 monclient: get_auth_request con 0x7fd0d8081310 auth_method 0 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.078+0000 7fd107fff640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7fd0d8081310 0x7fd0d8083730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.078+0000 7fd0e3fff640 10 client.5304.objecter ms_handle_connect 0x7fd0d8081310 2026-02-24T18:40:47.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.087+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== osd.2 v2:10.20.193.110:6824/3650974883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd0f403fea0 con 0x7fd0d8081310 2026-02-24T18:40:47.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.087+0000 7fd0e3fff640 10 client.5304.objecter ms_dispatch2 0x7fd108000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:47.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.087+0000 7fd0e3fff640 10 client.5304.objecter _finish_command 1 = system:0 2026-02-24T18:40:47.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.087+0000 7fd0e3fff640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 4 2026-02-24T18:40:47.089 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.087+0000 7fd0e3fff640 15 client.5304.objecter _session_command_op_remove 2 1 2026-02-24T18:40:47.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 10 client.5304.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:47.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 15 client.5304.objecter _session_command_op_assign -1 2 2026-02-24T18:40:47.095 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 20 client.5304.objecter _get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 20 client.5304.objecter _get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 15 client.5304.objecter _session_command_op_remove -1 2 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 20 client.5304.objecter get_session s=0x7fd0d8080c40 osd=2 3 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 15 client.5304.objecter _session_command_op_assign 2 2 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 4 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 10 client.5304.objecter _send_command 2 2026-02-24T18:40:47.096 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.093+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 --> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd108159e20 con 0x7fd0d8081310 2026-02-24T18:40:47.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.100+0000 7fd0e3fff640 1 -- 10.20.193.18:0/221898258 <== osd.2 v2:10.20.193.110:6824/3650974883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd108159e20 con 0x7fd0d8081310 2026-02-24T18:40:47.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.100+0000 7fd0e3fff640 10 client.5304.objecter ms_dispatch2 0x7fd108000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:47.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.100+0000 7fd0e3fff640 10 client.5304.objecter _finish_command 2 = system:0 2026-02-24T18:40:47.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.100+0000 7fd0e3fff640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 4 2026-02-24T18:40:47.102 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.100+0000 7fd0e3fff640 15 client.5304.objecter _session_command_op_remove 2 2 2026-02-24T18:40:47.103 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:47.103 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:47.103 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:47.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 10 librados: watch_flush enter 2026-02-24T18:40:47.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 10 librados: watch_flush exit 2026-02-24T18:40:47.104 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 10 client.5304.objecter close_session for osd.2 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7fd0d8081310 msgr2=0x7fd0d8083730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.110:6824/3650974883,v1:10.20.193.110:6825/3650974883] conn(0x7fd0d8081310 0x7fd0d8083730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 20 client.5304.objecter put_session s=0x7fd0d8080c40 osd=2 1 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 20 client.5304.objecter shutdown clearing up homeless session... 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 10 client.5304.objecter successfully canceled tick 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0d8078e30 msgr2=0x7fd0d807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd0d8078e30 0x7fd0d807b2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1081540a0 tx=0x7fd0f8007ef0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 10 monclient: shutdown 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 msgr2=0x7fd1081f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 --2- 10.20.193.18:0/221898258 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1080704c0 0x7fd1081f19e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402f430 tx=0x7fd0f40047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.105 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd107fff640 1 -- 10.20.193.18:0/221898258 reap_dead start 2026-02-24T18:40:47.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 shutdown_connections 2026-02-24T18:40:47.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 >> 10.20.193.18:0/221898258 conn(0x7fd1080cd600 msgr2=0x7fd108157930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 shutdown_connections 2026-02-24T18:40:47.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 -- 10.20.193.18:0/221898258 wait complete. 2026-02-24T18:40:47.106 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.101+0000 7fd10dadb640 1 librados: shutdown 2026-02-24T18:40:47.111 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-24T18:40:47.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 10 monclient: get_monmap_and_config 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 10 monclient: monmap: 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:47.181086+0000 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:47.181086+0000 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:47.182 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:47.183 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:47.183 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:47.183 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:47.183 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 1 Processor -- start 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 1 -- start start 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.180+0000 7f1a89b03640 10 monclient: init 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient: _add_conns ranks=[0,5,4,2,1,3] 2026-02-24T18:40:47.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1a841527d0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f1a84152bb0 0x7f1a840c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-c con 0x7f1a84152bb0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f1a840c5fc0 0x7f1a840c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-b con 0x7f1a840c5fc0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8406e300 con 0x7f1a841527d0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8406f1f0 con 0x7f1a840c5fc0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1a840c6470 con 0x7f1a84152bb0 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a89b03640 10 monclient(hunting): authenticate will time out at 1790.862400s 2026-02-24T18:40:47.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a837fe640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f1a841527d0 0x7f1a8406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48682/0 (socket says 10.20.193.18:48682) 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a837fe640 1 -- 10.20.193.18:0/877522999 learned_addr learned my addr 10.20.193.18:0/877522999 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a82ffd640 1 --2- 10.20.193.18:0/877522999 >> v2:10.20.193.110:3300/0 conn(0x7f1a84152bb0 0x7f1a840c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.181+0000 7f1a83fff640 1 --2- 10.20.193.18:0/877522999 >> v2:10.20.193.32:3300/0 conn(0x7f1a840c5fc0 0x7f1a840c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): get_auth_request con 0x7f1a84152bb0 auth_method 0 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): get_auth_request con 0x7f1a840c5fc0 auth_method 0 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.182+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 1 -- 10.20.193.18:0/877522999 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998108022 0 0) 0x7f1a8406e300 con 0x7f1a841527d0 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 10 monclient(hunting): _negotiate 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 10 monclient(hunting): my global_id is 5307 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a827fc640 1 -- 10.20.193.18:0/877522999 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a68003620 con 0x7f1a841527d0 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 10 monclient(hunting): handle_auth_done global_id 5233 payload 274 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 1 -- 10.20.193.18:0/877522999 >> v2:10.20.193.110:3300/0 conn(0x7f1a84152bb0 msgr2=0x7f1a840c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 1 --2- 10.20.193.18:0/877522999 >> v2:10.20.193.110:3300/0 conn(0x7f1a84152bb0 0x7f1a840c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 1 -- 10.20.193.18:0/877522999 >> v1:10.20.193.18:6789/0 conn(0x7f1a841527d0 legacy=0x7f1a8406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 1 monclient: found mon.noname-b 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.183+0000 7f1a83fff640 1 -- 10.20.193.18:0/877522999 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8406e5e0 con 0x7f1a840c5fc0 2026-02-24T18:40:47.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.184+0000 7f1a83fff640 1 --2- 10.20.193.18:0/877522999 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a840c5fc0 0x7f1a840c8e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a840750e0 tx=0x7f1a7802f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff73a2f6040e883 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 1 -- 10.20.193.18:0/877522999 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7803c070 con 0x7f1a840c5fc0 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 10 monclient: dump: 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.196 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a89b03640 5 monclient: authenticate success, global_id 5233 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.185+0000 7f1a89b03640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:47.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a827fc640 1 -- 10.20.193.18:0/877522999 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a78037440 con 0x7f1a840c5fc0 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a827fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 4 set_mon_vals no callback set 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 10 monclient: shutdown 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 -- 10.20.193.18:0/877522999 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a840c5fc0 msgr2=0x7f1a840c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 --2- 10.20.193.18:0/877522999 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a840c5fc0 0x7f1a840c8e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a840750e0 tx=0x7f1a7802f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 -- 10.20.193.18:0/877522999 shutdown_connections 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 --2- 10.20.193.18:0/877522999 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a840c5fc0 0x7f1a840c8e60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 --2- 10.20.193.18:0/877522999 >> v2:10.20.193.110:3300/0 conn(0x7f1a84152bb0 0x7f1a840c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 -- 10.20.193.18:0/877522999 >> 10.20.193.18:0/877522999 conn(0x7f1a840cd780 msgr2=0x7f1a840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 -- 10.20.193.18:0/877522999 shutdown_connections 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 1 -- 10.20.193.18:0/877522999 wait complete. 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.186+0000 7f1a89b03640 10 monclient: monmap: 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:47.199 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 librados: starting msgr at 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 librados: starting objecter 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 Processor -- start 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 -- start start 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 librados: setting wanted keys 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 librados: calling monclient init 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient: init 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-c con 0x7f1a840c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a841527d0 0x7f1a840ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-a con 0x7f1a841527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a84152bb0 0x7f1a84157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): picked mon.noname-b con 0x7f1a84152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8406ee80 con 0x7f1a841527d0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8406eba0 con 0x7f1a84152bb0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a840c93a0 con 0x7f1a840c5fc0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a89b03640 10 monclient(hunting): authenticate will time out at 1790.868653s 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a83fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a84152bb0 0x7f1a84157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.187+0000 7f1a837fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a841527d0 0x7f1a840ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:37992/0 (socket says 10.20.193.18:37992) 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 1 -- 10.20.193.18:0/3371716053 learned_addr learned my addr 10.20.193.18:0/3371716053 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): get_auth_request con 0x7f1a840c5fc0 auth_method 0 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): get_auth_request con 0x7f1a841527d0 auth_method 0 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): get_auth_request con 0x7f1a84152bb0 auth_method 0 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.202 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.188+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 10 monclient(hunting): handle_auth_done global_id 5222 payload 824 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 -- 10.20.193.18:0/3371716053 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a84152bb0 msgr2=0x7f1a84157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.203 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a84152bb0 0x7f1a84157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 -- 10.20.193.18:0/3371716053 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a841527d0 msgr2=0x7f1a840ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a841527d0 0x7f1a840ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 monclient: found mon.noname-c 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.189+0000 7f1a837fe640 1 -- 10.20.193.18:0/3371716053 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a840bb470 con 0x7f1a840c5fc0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.190+0000 7f1a837fe640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c760 tx=0x7f1a7000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.190+0000 7f1a5ffff640 10 client.?.objecter ms_handle_connect 0x7f1a840c5fc0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.190+0000 7f1a5ffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.191+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70017070 con 0x7f1a840c5fc0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.191+0000 7f1a5ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.191+0000 7f1a5ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:47.204 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: dump: 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 5 monclient: authenticate success, global_id 5222 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _renew_subs 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a840bb750 con 0x7f1a840c5fc0 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 client.5222.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _renew_subs 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a841f2830 con 0x7f1a840c5fc0 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 1 librados: init done 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:47.206 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a841f2370 con 0x7f1a840c5fc0 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70005bb0 con 0x7f1a840c5fc0 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70021060 con 0x7f1a840c5fc0 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a5ffff640 10 monclient: dump: 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.207 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.208 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.208 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.208 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.208 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.192+0000 7f1a81ffb640 4 set_mon_vals no callback set 2026-02-24T18:40:47.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.299+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1a70024080 con 0x7f1a840c5fc0 2026-02-24T18:40:47.301 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.300+0000 7f1a5ffff640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 0x7f1a5407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.300+0000 7f1a82ffd640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 0x7f1a5407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.300+0000 7f1a82ffd640 10 monclient: get_auth_request con 0x7f1a54078e30 auth_method 0 2026-02-24T18:40:47.302 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.300+0000 7f1a5ffff640 10 client.5222.objecter ms_handle_connect 0x7f1a54078e30 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.300+0000 7f1a82ffd640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 0x7f1a5407b2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1a840c4270 tx=0x7f1a74009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.301+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f1a7005d540 con 0x7f1a840c5fc0 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.301+0000 7f1a5ffff640 10 client.5222.objecter ms_dispatch2 0x7f1a84000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.301+0000 7f1a5ffff640 3 client.5222.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.301+0000 7f1a5ffff640 3 client.5222.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:47.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.301+0000 7f1a5ffff640 20 client.5222.objecter dump_active .. 0 homeless 2026-02-24T18:40:47.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.307+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1a841f2370 con 0x7f1a840c5fc0 2026-02-24T18:40:47.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.307+0000 7f1a5ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.307+0000 7f1a5ffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:47.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.390+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1a840bba30 con 0x7f1a54078e30 2026-02-24T18:40:47.392 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.390+0000 7f1a5ffff640 1 -- 10.20.193.18:0/3371716053 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1a840bba30 con 0x7f1a54078e30 2026-02-24T18:40:47.392 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 2 to deep-scrub 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 10 librados: watch_flush enter 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 10 librados: watch_flush exit 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 20 client.5222.objecter shutdown clearing up homeless session... 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 10 client.5222.objecter successfully canceled tick 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 msgr2=0x7f1a5407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 0x7f1a5407b2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1a840c4270 tx=0x7f1a74009290 comp rx=0 tx=0).stop 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 10 monclient: shutdown 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.394 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 msgr2=0x7f1a840bad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c760 tx=0x7f1a7000cc30 comp rx=0 tx=0).stop 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 shutdown_connections 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f1a54078e30 0x7f1a5407b2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f1a84152bb0 0x7f1a84157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f1a841527d0 0x7f1a840ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 --2- 10.20.193.18:0/3371716053 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f1a840c5fc0 0x7f1a840bad10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 >> 10.20.193.18:0/3371716053 conn(0x7f1a840cd780 msgr2=0x7f1a840c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 shutdown_connections 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 -- 10.20.193.18:0/3371716053 wait complete. 2026-02-24T18:40:47.395 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.391+0000 7f1a89b03640 1 librados: shutdown 2026-02-24T18:40:47.401 INFO:tasks.ceph:Scrubbing osd.6 2026-02-24T18:40:47.401 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 10 monclient: get_monmap_and_config 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 10 monclient: monmap: 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:47.471992+0000 2026-02-24T18:40:47.473 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:47.471992+0000 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 1 Processor -- start 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 1 -- start start 2026-02-24T18:40:47.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.471+0000 7f7d2e3d5640 10 monclient: init 2026-02-24T18:40:47.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient: _add_conns ranks=[4,1,3,0,5,2] 2026-02-24T18:40:47.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7d281503e0 0x7f7d28159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-b con 0x7f7d281503e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7d2815a070 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7d280704c0 0x7f7d280c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-a con 0x7f7d280704c0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d2815a570 con 0x7f7d2815a070 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280c7d70 con 0x7f7d280704c0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280c8610 con 0x7f7d281503e0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2e3d5640 10 monclient(hunting): authenticate will time out at 1791.153219s 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7d280704c0 0x7f7d280c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.478 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d27fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7d281503e0 0x7f7d28159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7d280704c0 0x7f7d280c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49562/0 (socket says 10.20.193.18:49562) 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 1 -- 10.20.193.18:0/1670574079 learned_addr learned my addr 10.20.193.18:0/1670574079 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 10 monclient(hunting): get_auth_request con 0x7f7d280704c0 auth_method 0 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d2c94b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d27fff640 10 monclient(hunting): get_auth_request con 0x7f7d281503e0 auth_method 0 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d27fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.472+0000 7f7d27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.479 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.473+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_done global_id 5313 payload 274 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 -- 10.20.193.18:0/1670574079 >> v2:10.20.193.32:3300/0 conn(0x7f7d281503e0 msgr2=0x7f7d28159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 --2- 10.20.193.18:0/1670574079 >> v2:10.20.193.32:3300/0 conn(0x7f7d281503e0 0x7f7d28159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 -- 10.20.193.18:0/1670574079 >> v1:10.20.193.32:6789/0 conn(0x7f7d2815a070 legacy=0x7f7d2806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 -- 10.20.193.18:0/1670574079 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2815a850 con 0x7f7d280704c0 2026-02-24T18:40:47.480 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.474+0000 7f7d2c94b640 1 --2- 10.20.193.18:0/1670574079 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d280704c0 0x7f7d280c8aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d280ce490 tx=0x7f7d1c02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65e97adbef8fc9e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 1 -- 10.20.193.18:0/1670574079 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c02fd70 con 0x7f7d280704c0 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: dump: 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.487 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d2e3d5640 5 monclient: authenticate success, global_id 5313 2026-02-24T18:40:47.488 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d2e3d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 1 -- 10.20.193.18:0/1670574079 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0373f0 con 0x7f7d280704c0 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 1 -- 10.20.193.18:0/1670574079 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0377c0 con 0x7f7d280704c0 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d2e3d5640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d2e3d5640 4 set_mon_vals no callback set 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.476+0000 7f7d26ffd640 10 monclient: dump: 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.489 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 10 monclient: shutdown 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/1670574079 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d280704c0 msgr2=0x7f7d280c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/1670574079 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d280704c0 0x7f7d280c8aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d280ce490 tx=0x7f7d1c02f310 comp rx=0 tx=0).stop 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/1670574079 shutdown_connections 2026-02-24T18:40:47.490 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/1670574079 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d280704c0 0x7f7d280c8aa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/1670574079 >> v2:10.20.193.32:3300/0 conn(0x7f7d281503e0 0x7f7d28159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/1670574079 >> 10.20.193.18:0/1670574079 conn(0x7f7d280cd600 msgr2=0x7f7d280cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/1670574079 shutdown_connections 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/1670574079 wait complete. 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 10 monclient: monmap: 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.491 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 librados: starting msgr at 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 librados: starting objecter 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 Processor -- start 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 -- start start 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 librados: setting wanted keys 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 1 librados: calling monclient init 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.477+0000 7f7d2e3d5640 10 monclient: init 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:47.492 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d280704c0 0x7f7d281f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-b con 0x7f7d280704c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d281503e0 0x7f7d281f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-c con 0x7f7d281503e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 0x7f7d28147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): picked mon.noname-a con 0x7f7d2815a070 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281507c0 con 0x7f7d2815a070 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280c82a0 con 0x7f7d280704c0 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28150940 con 0x7f7d281503e0 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2e3d5640 10 monclient(hunting): authenticate will time out at 1791.159363s 2026-02-24T18:40:47.493 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 0x7f7d28147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d27fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d280704c0 0x7f7d281f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 0x7f7d28147fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49568/0 (socket says 10.20.193.18:49568) 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 1 -- 10.20.193.18:0/4180398885 learned_addr learned my addr 10.20.193.18:0/4180398885 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 10 monclient(hunting): get_auth_request con 0x7f7d2815a070 auth_method 0 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d2c94b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d277fe640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d281503e0 0x7f7d281f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d27fff640 10 monclient(hunting): get_auth_request con 0x7f7d280704c0 auth_method 0 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d27fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.494 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.478+0000 7f7d27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): get_auth_request con 0x7f7d281503e0 auth_method 0 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.495 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.479+0000 7f7d277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 10 monclient(hunting): handle_auth_done global_id 5316 payload 824 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 -- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d281503e0 msgr2=0x7f7d281f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7d281503e0 0x7f7d281f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 -- 10.20.193.18:0/4180398885 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d280704c0 msgr2=0x7f7d281f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7d280704c0 0x7f7d281f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.480+0000 7f7d2c94b640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d281f7600 con 0x7f7d2815a070 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.481+0000 7f7d2c94b640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 0x7f7d28147fb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009f50 tx=0x7f7d1c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.481+0000 7f7d24ff9640 10 client.?.objecter ms_handle_connect 0x7f7d2815a070 2026-02-24T18:40:47.496 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.481+0000 7f7d24ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c046020 con 0x7f7d2815a070 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 10 monclient: dump: 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.497 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.482+0000 7f7d24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 5 monclient: authenticate success, global_id 5316 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 monclient: _renew_subs 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1c041450 con 0x7f7d2815a070 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d281f78e0 con 0x7f7d2815a070 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009510 con 0x7f7d2815a070 2026-02-24T18:40:47.498 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d24ff9640 10 monclient: dump: 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.499 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d267fc640 4 set_mon_vals no callback set 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 client.5316.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 monclient: _renew_subs 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d28072980 con 0x7f7d2815a070 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 1 librados: init done 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 client.5316.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 15 client.5316.objecter _session_command_op_assign -1 1 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 20 client.5316.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 client.5316.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.500 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.483+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d28159b30 con 0x7f7d2815a070 2026-02-24T18:40:47.502 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.494+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7d1c041880 con 0x7f7d2815a070 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.495+0000 7f7d24ff9640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cf8078e30 0x7f7cf807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.495+0000 7f7d27fff640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cf8078e30 0x7f7cf807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.495+0000 7f7d27fff640 10 monclient: get_auth_request con 0x7f7cf8078e30 auth_method 0 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.495+0000 7f7d27fff640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cf8078e30 0x7f7cf807b2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009c40 tx=0x7f7d14009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.495+0000 7f7d24ff9640 10 client.5316.objecter ms_handle_connect 0x7f7cf8078e30 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7d1c0bd0c0 con 0x7f7d2815a070 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 10 client.5316.objecter ms_dispatch2 0x7f7d28000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 3 client.5316.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 3 client.5316.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 10 client.5316.objecter checking command 1 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7f7cf8081310 0x7f7cf8083730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.503 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter _get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 15 client.5316.objecter _session_command_op_remove -1 1 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter _get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 15 client.5316.objecter _session_command_op_assign 6 1 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 4 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 10 client.5316.objecter _send_command 1 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d1c03fe50 con 0x7f7cf8081310 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 20 client.5316.objecter dump_active .. 0 homeless 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c083c90 con 0x7f7d2815a070 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.496+0000 7f7d24ff9640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.497+0000 7f7d277fe640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7f7cf8081310 0x7f7cf8083730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.504 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.497+0000 7f7d277fe640 10 monclient: get_auth_request con 0x7f7cf8081310 auth_method 0 2026-02-24T18:40:47.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.499+0000 7f7d24ff9640 10 client.5316.objecter ms_handle_connect 0x7f7cf8081310 2026-02-24T18:40:47.505 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.499+0000 7f7d277fe640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7f7cf8081310 0x7f7cf8083730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.508+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== osd.6 v2:10.20.193.110:6800/3466222224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f7d1c03fe50 con 0x7f7cf8081310 2026-02-24T18:40:47.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.508+0000 7f7d24ff9640 10 client.5316.objecter ms_dispatch2 0x7f7d28000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:47.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.508+0000 7f7d24ff9640 10 client.5316.objecter _finish_command 1 = system:0 2026-02-24T18:40:47.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.508+0000 7f7d24ff9640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 4 2026-02-24T18:40:47.510 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.508+0000 7f7d24ff9640 15 client.5316.objecter _session_command_op_remove 6 1 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 10 client.5316.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 15 client.5316.objecter _session_command_op_assign -1 2 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 20 client.5316.objecter _get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 20 client.5316.objecter _get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 15 client.5316.objecter _session_command_op_remove -1 2 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 20 client.5316.objecter get_session s=0x7f7cf8080c40 osd=6 3 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 15 client.5316.objecter _session_command_op_assign 6 2 2026-02-24T18:40:47.516 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 4 2026-02-24T18:40:47.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 10 client.5316.objecter _send_command 2 2026-02-24T18:40:47.517 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.514+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 --> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7d2814ed70 con 0x7f7cf8081310 2026-02-24T18:40:47.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d24ff9640 1 -- 10.20.193.18:0/4180398885 <== osd.6 v2:10.20.193.110:6800/3466222224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7d2814ed70 con 0x7f7cf8081310 2026-02-24T18:40:47.522 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d24ff9640 10 client.5316.objecter ms_dispatch2 0x7f7d28000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d24ff9640 10 client.5316.objecter _finish_command 2 = system:0 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d24ff9640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 4 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d24ff9640 15 client.5316.objecter _session_command_op_remove 6 2 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:47.523 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 10 librados: watch_flush enter 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 10 librados: watch_flush exit 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 10 client.5316.objecter close_session for osd.6 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7f7cf8081310 msgr2=0x7f7cf8083730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.110:6800/3466222224,v1:10.20.193.110:6801/3466222224] conn(0x7f7cf8081310 0x7f7cf8083730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 20 client.5316.objecter put_session s=0x7f7cf8080c40 osd=6 1 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 20 client.5316.objecter shutdown clearing up homeless session... 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 10 client.5316.objecter successfully canceled tick 2026-02-24T18:40:47.525 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cf8078e30 msgr2=0x7f7cf807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7cf8078e30 0x7f7cf807b2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009c40 tx=0x7f7d14009290 comp rx=0 tx=0).stop 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 10 monclient: shutdown 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 msgr2=0x7f7d28147fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 --2- 10.20.193.18:0/4180398885 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7d2815a070 0x7f7d28147fb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009f50 tx=0x7f7d1c004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2c94b640 1 -- 10.20.193.18:0/4180398885 reap_dead start 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 shutdown_connections 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 >> 10.20.193.18:0/4180398885 conn(0x7f7d280cd600 msgr2=0x7f7d280cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 shutdown_connections 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 -- 10.20.193.18:0/4180398885 wait complete. 2026-02-24T18:40:47.526 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.521+0000 7f7d2e3d5640 1 librados: shutdown 2026-02-24T18:40:47.531 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-24T18:40:47.600 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 10 monclient: get_monmap_and_config 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 10 monclient: monmap: 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:47.600192+0000 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:47.600192+0000 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.601 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:47.602 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 1 Processor -- start 2026-02-24T18:40:47.604 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 1 -- start start 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.599+0000 7f46b98fa640 10 monclient: init 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient: _add_conns ranks=[4,3,1,5,0,2] 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f46b41527d0 0x7f46b406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-b con 0x7f46b41527d0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 0x7f46b40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-a con 0x7f46b4152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f46b40c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b406e300 con 0x7f46b40c5fc0 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f46b406f1f0 con 0x7f46b4152bb0 2026-02-24T18:40:47.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f46b40c6470 con 0x7f46b41527d0 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b98fa640 10 monclient(hunting): authenticate will time out at 1791.281332s 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b2ffd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f46b41527d0 0x7f46b406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 0x7f46b40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 0x7f46b40c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49574/0 (socket says 10.20.193.18:49574) 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 1 -- 10.20.193.18:0/2474752792 learned_addr learned my addr 10.20.193.18:0/2474752792 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 10 monclient(hunting): get_auth_request con 0x7f46b4152bb0 auth_method 0 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b2ffd640 10 monclient(hunting): get_auth_request con 0x7f46b41527d0 auth_method 0 2026-02-24T18:40:47.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.600+0000 7f46b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.601+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_done global_id 5245 payload 274 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 1 -- 10.20.193.18:0/2474752792 >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 msgr2=0x7f46b40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 1 --2- 10.20.193.18:0/2474752792 >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 0x7f46b40c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 1 -- 10.20.193.18:0/2474752792 >> v1:10.20.193.32:6789/0 conn(0x7f46b40c5fc0 legacy=0x7f46b40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 1 monclient: found mon.noname-b 2026-02-24T18:40:47.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.602+0000 7f46b2ffd640 1 -- 10.20.193.18:0/2474752792 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b406e5e0 con 0x7f46b41527d0 2026-02-24T18:40:47.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.603+0000 7f46b2ffd640 1 --2- 10.20.193.18:0/2474752792 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b406da60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46b40caec0 tx=0x7f46a802ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f736f63a981a11 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 1 -- 10.20.193.18:0/2474752792 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a803c070 con 0x7f46b41527d0 2026-02-24T18:40:47.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 10 monclient: dump: 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b98fa640 5 monclient: authenticate success, global_id 5245 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.604+0000 7f46b98fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 1 -- 10.20.193.18:0/2474752792 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a802fa50 con 0x7f46b41527d0 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 1 -- 10.20.193.18:0/2474752792 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a802fe20 con 0x7f46b41527d0 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b1ffb640 10 monclient: dump: 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.614 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 4 set_mon_vals no callback set 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 10 monclient: shutdown 2026-02-24T18:40:47.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 -- 10.20.193.18:0/2474752792 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 msgr2=0x7f46b406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 --2- 10.20.193.18:0/2474752792 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b406da60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46b40caec0 tx=0x7f46a802ede0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 -- 10.20.193.18:0/2474752792 shutdown_connections 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 --2- 10.20.193.18:0/2474752792 >> v2:10.20.193.18:3300/0 conn(0x7f46b4152bb0 0x7f46b40c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 --2- 10.20.193.18:0/2474752792 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b406da60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 -- 10.20.193.18:0/2474752792 >> 10.20.193.18:0/2474752792 conn(0x7f46b40cd780 msgr2=0x7f46b40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 -- 10.20.193.18:0/2474752792 shutdown_connections 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.605+0000 7f46b98fa640 1 -- 10.20.193.18:0/2474752792 wait complete. 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 10 monclient: monmap: 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.616 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 librados: starting msgr at 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 librados: starting objecter 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 Processor -- start 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 -- start start 2026-02-24T18:40:47.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 librados: setting wanted keys 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 1 librados: calling monclient init 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.606+0000 7f46b98fa640 10 monclient: init 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 0x7f46b4155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-a con 0x7f46b40c5fc0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b4145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-b con 0x7f46b41527d0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f46b4152bb0 0x7f46b41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): picked mon.noname-c con 0x7f46b4152bb0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f46b406ee80 con 0x7f46b40c5fc0 2026-02-24T18:40:47.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46b406eba0 con 0x7f46b41527d0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46b406dfa0 con 0x7f46b4152bb0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b98fa640 10 monclient(hunting): authenticate will time out at 1791.288328s 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 0x7f46b4155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b27fc640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b4145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b37fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f46b4152bb0 0x7f46b41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 0x7f46b4155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49588/0 (socket says 10.20.193.18:49588) 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 1 -- 10.20.193.18:0/3045428628 learned_addr learned my addr 10.20.193.18:0/3045428628 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 10 monclient(hunting): get_auth_request con 0x7f46b40c5fc0 auth_method 0 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.619 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b27fc640 10 monclient(hunting): get_auth_request con 0x7f46b41527d0 auth_method 0 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.607+0000 7f46b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): get_auth_request con 0x7f46b4152bb0 auth_method 0 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.608+0000 7f46b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 10 monclient(hunting): handle_auth_done global_id 5248 payload 824 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 -- 10.20.193.18:0/3045428628 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f46b4152bb0 msgr2=0x7f46b41f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f46b4152bb0 0x7f46b41f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 -- 10.20.193.18:0/3045428628 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 msgr2=0x7f46b4155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 0x7f46b4155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 monclient: found mon.noname-b 2026-02-24T18:40:47.621 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.609+0000 7f46b27fc640 1 -- 10.20.193.18:0/3045428628 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41560c0 con 0x7f46b41527d0 2026-02-24T18:40:47.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.610+0000 7f46b27fc640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b4145e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e7d0 tx=0x7f469c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.610+0000 7f468bfff640 10 client.?.objecter ms_handle_connect 0x7f46b41527d0 2026-02-24T18:40:47.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.610+0000 7f468bfff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c017070 con 0x7f46b41527d0 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: dump: 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.624 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c00c940 con 0x7f46b41527d0 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00ccf0 con 0x7f46b41527d0 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:47.625 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b17fa640 4 set_mon_vals no callback set 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f468bfff640 10 monclient: dump: 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 5 monclient: authenticate success, global_id 5248 2026-02-24T18:40:47.626 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _renew_subs 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b4154800 con 0x7f46b41527d0 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 client.5248.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _renew_subs 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b4154dc0 con 0x7f46b41527d0 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 1 librados: init done 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:47.627 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.612+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b406da60 con 0x7f46b41527d0 2026-02-24T18:40:47.628 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.624+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f469c024060 con 0x7f46b41527d0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.624+0000 7f468bfff640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 0x7f468407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.624+0000 7f46b2ffd640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 0x7f468407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.624+0000 7f46b2ffd640 10 monclient: get_auth_request con 0x7f4684078e00 auth_method 0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.625+0000 7f468bfff640 10 client.5248.objecter ms_handle_connect 0x7f4684078e00 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.625+0000 7f46b2ffd640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 0x7f468407b2c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f46a802ee20 tx=0x7f46a8033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.626+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f469c05e000 con 0x7f46b41527d0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.626+0000 7f468bfff640 10 client.5248.objecter ms_dispatch2 0x7f46b4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.626+0000 7f468bfff640 3 client.5248.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.626+0000 7f468bfff640 3 client.5248.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:47.629 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.626+0000 7f468bfff640 20 client.5248.objecter dump_active .. 0 homeless 2026-02-24T18:40:47.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.723+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f469c061fc0 con 0x7f46b41527d0 2026-02-24T18:40:47.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.723+0000 7f468bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.725 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.723+0000 7f468bfff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:47.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.807+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f46b406d260 con 0x7f4684078e00 2026-02-24T18:40:47.809 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.807+0000 7f468bfff640 1 -- 10.20.193.18:0/3045428628 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f46b406d260 con 0x7f4684078e00 2026-02-24T18:40:47.809 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 6 to deep-scrub 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 10 librados: watch_flush enter 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 10 librados: watch_flush exit 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 20 client.5248.objecter shutdown clearing up homeless session... 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 10 client.5248.objecter successfully canceled tick 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 msgr2=0x7f468407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 0x7f468407b2c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f46a802ee20 tx=0x7f46a8033000 comp rx=0 tx=0).stop 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 10 monclient: shutdown 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 msgr2=0x7f46b4145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.811 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b4145e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e7d0 tx=0x7f469c00eca0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 shutdown_connections 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4684078e00 0x7f468407b2c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f46b41527d0 0x7f46b4145e20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f46b4152bb0 0x7f46b41f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 --2- 10.20.193.18:0/3045428628 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f46b40c5fc0 0x7f46b4155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 >> 10.20.193.18:0/3045428628 conn(0x7f46b40cd780 msgr2=0x7f46b406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 shutdown_connections 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 -- 10.20.193.18:0/3045428628 wait complete. 2026-02-24T18:40:47.812 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.808+0000 7f46b98fa640 1 librados: shutdown 2026-02-24T18:40:47.818 INFO:tasks.ceph:Scrubbing osd.10 2026-02-24T18:40:47.819 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:47.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.888+0000 7f4c42f7e640 10 monclient: get_monmap_and_config 2026-02-24T18:40:47.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.888+0000 7f4c42f7e640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.888+0000 7f4c42f7e640 10 monclient: monmap: 2026-02-24T18:40:47.890 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.890 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:47.889602+0000 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:47.889602+0000 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:47.891 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 Processor -- start 2026-02-24T18:40:47.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 -- start start 2026-02-24T18:40:47.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient: init 2026-02-24T18:40:47.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient: _add_conns ranks=[5,1,3,4,2,0] 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4c3c1503e0 0x7f4c3c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-c con 0x7f4c3c1503e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4c3c15a070 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4c3c0704c0 0x7f4c3c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-a con 0x7f4c3c0704c0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3c15a570 con 0x7f4c3c15a070 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3c0c7d70 con 0x7f4c3c0704c0 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3c0c8610 con 0x7f4c3c1503e0 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.889+0000 7f4c42f7e640 10 monclient(hunting): authenticate will time out at 1791.570779s 2026-02-24T18:40:47.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f4c3c0704c0 0x7f4c3c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f4c3c1503e0 0x7f4c3c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c33fff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f4c3c15a070 0x7f4c3c06fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:34816/0 (socket says 10.20.193.18:34816) 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c33fff640 1 -- 10.20.193.18:0/4242080206 learned_addr learned my addr 10.20.193.18:0/4242080206 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): get_auth_request con 0x7f4c3c0704c0 auth_method 0 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): get_auth_request con 0x7f4c3c1503e0 auth_method 0 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.897 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.890+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 1 -- 10.20.193.18:0/4242080206 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848620545 0 0) 0x7f4c3c15a570 con 0x7f4c3c15a070 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 10 monclient(hunting): _negotiate 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 10 monclient(hunting): my global_id is 5251 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c337fe640 1 -- 10.20.193.18:0/4242080206 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c28003620 con 0x7f4c3c15a070 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 10 monclient(hunting): handle_auth_done global_id 5325 payload 274 2026-02-24T18:40:47.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 1 -- 10.20.193.18:0/4242080206 >> v2:10.20.193.110:3300/0 conn(0x7f4c3c1503e0 msgr2=0x7f4c3c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 1 --2- 10.20.193.18:0/4242080206 >> v2:10.20.193.110:3300/0 conn(0x7f4c3c1503e0 0x7f4c3c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 1 -- 10.20.193.18:0/4242080206 >> v1:10.20.193.32:6789/0 conn(0x7f4c3c15a070 legacy=0x7f4c3c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.891+0000 7f4c414f4640 1 -- 10.20.193.18:0/4242080206 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c15a850 con 0x7f4c3c0704c0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.892+0000 7f4c414f4640 1 --2- 10.20.193.18:0/4242080206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c0c8aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0ce490 tx=0x7f4c3802f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0be0e2cf834348e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 1 -- 10.20.193.18:0/4242080206 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3803c070 con 0x7f4c3c0704c0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 10 monclient: dump: 2026-02-24T18:40:47.899 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c42f7e640 5 monclient: authenticate success, global_id 5325 2026-02-24T18:40:47.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.893+0000 7f4c42f7e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:47.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 1 -- 10.20.193.18:0/4242080206 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38037440 con 0x7f4c3c0704c0 2026-02-24T18:40:47.906 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 1 -- 10.20.193.18:0/4242080206 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c380377f0 con 0x7f4c3c0704c0 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 4 set_mon_vals no callback set 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c337fe640 10 monclient: dump: 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.907 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 10 monclient: shutdown 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 -- 10.20.193.18:0/4242080206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c0704c0 msgr2=0x7f4c3c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/4242080206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c0c8aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0ce490 tx=0x7f4c3802f250 comp rx=0 tx=0).stop 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 -- 10.20.193.18:0/4242080206 shutdown_connections 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/4242080206 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c0c8aa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/4242080206 >> v2:10.20.193.110:3300/0 conn(0x7f4c3c1503e0 0x7f4c3c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.908 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 -- 10.20.193.18:0/4242080206 >> 10.20.193.18:0/4242080206 conn(0x7f4c3c0cd600 msgr2=0x7f4c3c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 -- 10.20.193.18:0/4242080206 shutdown_connections 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.894+0000 7f4c42f7e640 1 -- 10.20.193.18:0/4242080206 wait complete. 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient: build_initial_monmap 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient: monmap: 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:47.909 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 librados: starting msgr at 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 librados: starting objecter 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 Processor -- start 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 -- start start 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 librados: setting wanted keys 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 librados: calling monclient init 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient: init 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.910 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-c con 0x7f4c3c0704c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4c3c1503e0 0x7f4c3c1f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-b con 0x7f4c3c1503e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 0x7f4c3c0ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): picked mon.noname-a con 0x7f4c3c15a070 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1507c0 con 0x7f4c3c15a070 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c0c82a0 con 0x7f4c3c1503e0 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c150940 con 0x7f4c3c0704c0 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c42f7e640 10 monclient(hunting): authenticate will time out at 1791.576741s 2026-02-24T18:40:47.911 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.895+0000 7f4c414f4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 0x7f4c3c0ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4c3c1503e0 0x7f4c3c1f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 0x7f4c3c0ba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49596/0 (socket says 10.20.193.18:49596) 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 1 -- 10.20.193.18:0/1152487920 learned_addr learned my addr 10.20.193.18:0/1152487920 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): get_auth_request con 0x7f4c3c15a070 auth_method 0 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): get_auth_request con 0x7f4c3c0704c0 auth_method 0 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.912 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): get_auth_request con 0x7f4c3c1503e0 auth_method 0 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c414f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.913 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c40cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.896+0000 7f4c33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 10 monclient(hunting): handle_auth_done global_id 5328 payload 824 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 -- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4c3c0704c0 msgr2=0x7f4c3c1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f4c3c0704c0 0x7f4c3c1f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 -- 10.20.193.18:0/1152487920 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4c3c1503e0 msgr2=0x7f4c3c1f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f4c3c1503e0 0x7f4c3c1f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 10 monclient: _finish_hunting 0 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 monclient: found mon.noname-a 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.914 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.897+0000 7f4c414f4640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c1f3d10 con 0x7f4c3c15a070 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.898+0000 7f4c414f4640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 0x7f4c3c0ba300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009e90 tx=0x7f4c38004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.898+0000 7f4c317fa640 10 client.?.objecter ms_handle_connect 0x7f4c3c15a070 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.898+0000 7f4c317fa640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3803c050 con 0x7f4c3c15a070 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: dump: 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.915 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: _finish_auth 0 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: _check_auth_tickets 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38048070 con 0x7f4c3c15a070 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38009450 con 0x7f4c3c15a070 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:47.916 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c317fa640 10 monclient: dump: 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c32ffd640 4 set_mon_vals no callback set 2026-02-24T18:40:47.917 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 5 monclient: authenticate success, global_id 5328 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 10 monclient: _renew_subs 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c1f3ff0 con 0x7f4c3c15a070 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 10 client.5328.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 10 monclient: _renew_subs 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c06d620 con 0x7f4c3c15a070 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.900+0000 7f4c42f7e640 1 librados: init done 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 10 client.5328.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 15 client.5328.objecter _session_command_op_assign -1 1 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 20 client.5328.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 10 client.5328.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:47.918 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:47.919 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.901+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c3c159b30 con 0x7f4c3c15a070 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.912+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4c38004280 con 0x7f4c3c15a070 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.912+0000 7f4c317fa640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4c0c078e50 0x7f4c0c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.912+0000 7f4c40cf3640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4c0c078e50 0x7f4c0c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.912+0000 7f4c40cf3640 10 monclient: get_auth_request con 0x7f4c0c078e50 auth_method 0 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.913+0000 7f4c40cf3640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4c0c078e50 0x7f4c0c07b310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009c40 tx=0x7f4c2c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.913+0000 7f4c317fa640 10 client.5328.objecter ms_handle_connect 0x7f4c0c078e50 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f4c380840c0 con 0x7f4c3c15a070 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 10 client.5328.objecter ms_dispatch2 0x7f4c3c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 3 client.5328.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:47.921 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 3 client.5328.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 10 client.5328.objecter checking command 1 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f4c0c081390 0x7f4c0c0837b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter _get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 15 client.5328.objecter _session_command_op_remove -1 1 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter _get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 15 client.5328.objecter _session_command_op_assign 10 1 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 4 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 10 client.5328.objecter _send_command 1 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c3803bea0 con 0x7f4c0c081390 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 20 client.5328.objecter dump_active .. 0 homeless 2026-02-24T18:40:47.922 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c38084480 con 0x7f4c3c15a070 2026-02-24T18:40:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c317fa640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.914+0000 7f4c33fff640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f4c0c081390 0x7f4c0c0837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.915+0000 7f4c33fff640 10 monclient: get_auth_request con 0x7f4c0c081390 auth_method 0 2026-02-24T18:40:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.916+0000 7f4c317fa640 10 client.5328.objecter ms_handle_connect 0x7f4c0c081390 2026-02-24T18:40:47.923 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.916+0000 7f4c33fff640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f4c0c081390 0x7f4c0c0837b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.926+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== osd.10 v2:10.20.193.110:6808/4149932150 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4c3803bea0 con 0x7f4c0c081390 2026-02-24T18:40:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.926+0000 7f4c317fa640 10 client.5328.objecter ms_dispatch2 0x7f4c3c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.926+0000 7f4c317fa640 10 client.5328.objecter _finish_command 1 = system:0 2026-02-24T18:40:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.926+0000 7f4c317fa640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 4 2026-02-24T18:40:47.928 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.926+0000 7f4c317fa640 15 client.5328.objecter _session_command_op_remove 10 1 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 10 client.5328.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 15 client.5328.objecter _session_command_op_assign -1 2 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 20 client.5328.objecter _get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 20 client.5328.objecter _get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 15 client.5328.objecter _session_command_op_remove -1 2 2026-02-24T18:40:47.934 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 20 client.5328.objecter get_session s=0x7f4c0c080cc0 osd=10 3 2026-02-24T18:40:47.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 15 client.5328.objecter _session_command_op_assign 10 2 2026-02-24T18:40:47.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 4 2026-02-24T18:40:47.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 10 client.5328.objecter _send_command 2 2026-02-24T18:40:47.935 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.932+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 --> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4c3c06dbe0 con 0x7f4c0c081390 2026-02-24T18:40:47.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.946+0000 7f4c317fa640 1 -- 10.20.193.18:0/1152487920 <== osd.10 v2:10.20.193.110:6808/4149932150 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7f4c3c06dbe0 con 0x7f4c0c081390 2026-02-24T18:40:47.948 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.946+0000 7f4c317fa640 10 client.5328.objecter ms_dispatch2 0x7f4c3c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.946+0000 7f4c317fa640 10 client.5328.objecter _finish_command 2 = system:0 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.946+0000 7f4c317fa640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 4 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.946+0000 7f4c317fa640 15 client.5328.objecter _session_command_op_remove 10 2 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:47.949 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 10 librados: watch_flush enter 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 10 librados: watch_flush exit 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 10 client.5328.objecter close_session for osd.10 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f4c0c081390 msgr2=0x7f4c0c0837b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.110:6808/4149932150,v1:10.20.193.110:6809/4149932150] conn(0x7f4c0c081390 0x7f4c0c0837b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 20 client.5328.objecter put_session s=0x7f4c0c080cc0 osd=10 1 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 20 client.5328.objecter shutdown clearing up homeless session... 2026-02-24T18:40:47.951 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 10 client.5328.objecter successfully canceled tick 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4c0c078e50 msgr2=0x7f4c0c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f4c0c078e50 0x7f4c0c07b310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009c40 tx=0x7f4c2c009290 comp rx=0 tx=0).stop 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 10 monclient: shutdown 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 msgr2=0x7f4c3c0ba300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 --2- 10.20.193.18:0/1152487920 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f4c3c15a070 0x7f4c3c0ba300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009e90 tx=0x7f4c38004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c414f4640 1 -- 10.20.193.18:0/1152487920 reap_dead start 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 shutdown_connections 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 >> 10.20.193.18:0/1152487920 conn(0x7f4c3c0cd600 msgr2=0x7f4c3c14eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 shutdown_connections 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 -- 10.20.193.18:0/1152487920 wait complete. 2026-02-24T18:40:47.952 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:47.947+0000 7f4c42f7e640 1 librados: shutdown 2026-02-24T18:40:47.956 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2026-02-24T18:40:48.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.024+0000 7f9bf9f6f640 10 monclient: get_monmap_and_config 2026-02-24T18:40:48.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.024+0000 7f9bf9f6f640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.024+0000 7f9bf9f6f640 10 monclient: monmap: 2026-02-24T18:40:48.026 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.026 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:48.025706+0000 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:48.025706+0000 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:48.027 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 1 Processor -- start 2026-02-24T18:40:48.030 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 1 -- start start 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 10 monclient: init 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 10 monclient: _add_conns ranks=[2,5,4,3,0,1] 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9bf40c90d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.025+0000 7f9bf9f6f640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 0x7f9bf4158950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-c con 0x7f9bf40c94b0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9bf4158e90 0x7f9bf4074250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-b con 0x7f9bf4158e90 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf4073300 con 0x7f9bf40c90d0 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf415ce70 con 0x7f9bf4158e90 2026-02-24T18:40:48.031 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf4073540 con 0x7f9bf40c94b0 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf9f6f640 10 monclient(hunting): authenticate will time out at 1791.707029s 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 0x7f9bf4158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf3fff640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f9bf4158e90 0x7f9bf4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 0x7f9bf4158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:38040/0 (socket says 10.20.193.18:38040) 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 1 -- 10.20.193.18:0/1537607853 learned_addr learned my addr 10.20.193.18:0/1537607853 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): get_auth_request con 0x7f9bf40c94b0 auth_method 0 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf3fff640 10 monclient(hunting): get_auth_request con 0x7f9bf4158e90 auth_method 0 2026-02-24T18:40:48.032 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.026+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_done global_id 5257 payload 274 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 1 -- 10.20.193.18:0/1537607853 >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 msgr2=0x7f9bf4158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 1 --2- 10.20.193.18:0/1537607853 >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 0x7f9bf4158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 1 -- 10.20.193.18:0/1537607853 >> v1:10.20.193.110:6789/0 conn(0x7f9bf40c90d0 legacy=0x7f9bf4072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.027+0000 7f9bf3fff640 1 -- 10.20.193.18:0/1537607853 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4159440 con 0x7f9bf4158e90 2026-02-24T18:40:48.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.028+0000 7f9bf3fff640 1 --2- 10.20.193.18:0/1537607853 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf4158e90 0x7f9bf4074250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4072480 tx=0x7f9be802f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a92a22ab793d05 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 1 -- 10.20.193.18:0/1537607853 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be803c070 con 0x7f9bf4158e90 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 10 monclient: dump: 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.039 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf9f6f640 5 monclient: authenticate success, global_id 5257 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.029+0000 7f9bf9f6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 1 -- 10.20.193.18:0/1537607853 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be802fe30 con 0x7f9bf4158e90 2026-02-24T18:40:48.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 1 -- 10.20.193.18:0/1537607853 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037640 con 0x7f9bf4158e90 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf27fc640 10 monclient: dump: 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.041 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf9f6f640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.030+0000 7f9bf9f6f640 4 set_mon_vals no callback set 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 10 monclient: shutdown 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/1537607853 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf4158e90 msgr2=0x7f9bf4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/1537607853 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf4158e90 0x7f9bf4074250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4072480 tx=0x7f9be802f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/1537607853 shutdown_connections 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/1537607853 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf4158e90 0x7f9bf4074250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/1537607853 >> v2:10.20.193.110:3300/0 conn(0x7f9bf40c94b0 0x7f9bf4158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/1537607853 >> 10.20.193.18:0/1537607853 conn(0x7f9bf40cd780 msgr2=0x7f9bf40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/1537607853 shutdown_connections 2026-02-24T18:40:48.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/1537607853 wait complete. 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 10 monclient: monmap: 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 librados: starting msgr at 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 librados: starting objecter 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.031+0000 7f9bf9f6f640 1 Processor -- start 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 -- start start 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 librados: setting wanted keys 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 librados: calling monclient init 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient: init 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 0x7f9bf4148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-c con 0x7f9bf40c90d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 0x7f9bf41581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-b con 0x7f9bf40c94b0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9bf4158e90 0x7f9bf414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): picked mon.noname-a con 0x7f9bf4158e90 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4073a70 con 0x7f9bf4158e90 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4073790 con 0x7f9bf40c94b0 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4073d50 con 0x7f9bf40c90d0 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf9f6f640 10 monclient(hunting): authenticate will time out at 1791.713424s 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf3fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9bf4158e90 0x7f9bf414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.032+0000 7f9bf37fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 0x7f9bf4148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 0x7f9bf4148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:38054/0 (socket says 10.20.193.18:38054) 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 1 -- 10.20.193.18:0/3001434126 learned_addr learned my addr 10.20.193.18:0/3001434126 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): get_auth_request con 0x7f9bf40c90d0 auth_method 0 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 0x7f9bf41581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): get_auth_request con 0x7f9bf4158e90 auth_method 0 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): get_auth_request con 0x7f9bf40c94b0 auth_method 0 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.033+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 10 monclient(hunting): handle_auth_done global_id 5260 payload 824 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 -- 10.20.193.18:0/3001434126 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 msgr2=0x7f9bf4148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 0x7f9bf4148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 -- 10.20.193.18:0/3001434126 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9bf4158e90 msgr2=0x7f9bf414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9bf4158e90 0x7f9bf414ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.034+0000 7f9bf2ffd640 1 -- 10.20.193.18:0/3001434126 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4149330 con 0x7f9bf40c94b0 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.036+0000 7f9bf2ffd640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 0x7f9bf41581b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9be4006dc0 tx=0x7f9be400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.036+0000 7f9bcffff640 10 client.?.objecter ms_handle_connect 0x7f9bf40c94b0 2026-02-24T18:40:48.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.036+0000 7f9bcffff640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.038+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4017070 con 0x7f9bf40c94b0 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.038+0000 7f9bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.038+0000 7f9bcffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.038+0000 7f9bcffff640 10 monclient: dump: 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.050 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be400c910 con 0x7f9bf40c94b0 2026-02-24T18:40:48.051 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be400cce0 con 0x7f9bf40c94b0 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf1ffb640 4 set_mon_vals no callback set 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bcffff640 10 monclient: dump: 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.052 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 5 monclient: authenticate success, global_id 5260 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _renew_subs 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41495c0 con 0x7f9bf40c94b0 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 client.5260.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _renew_subs 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf4149b80 con 0x7f9bf40c94b0 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 1 librados: init done 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.039+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4156300 con 0x7f9bf40c94b0 2026-02-24T18:40:48.142 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.141+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9be4024060 con 0x7f9bf40c94b0 2026-02-24T18:40:48.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.141+0000 7f9bcffff640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 0x7f9bc407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.141+0000 7f9bf37fe640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 0x7f9bc407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.143 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.141+0000 7f9bf37fe640 10 monclient: get_auth_request con 0x7f9bc4078e50 auth_method 0 2026-02-24T18:40:48.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.142+0000 7f9bcffff640 10 client.5260.objecter ms_handle_connect 0x7f9bc4078e50 2026-02-24T18:40:48.144 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.142+0000 7f9bf37fe640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 0x7f9bc407b310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0004660 tx=0x7f9be0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.145+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f9be405e010 con 0x7f9bf40c94b0 2026-02-24T18:40:48.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.145+0000 7f9bcffff640 10 client.5260.objecter ms_dispatch2 0x7f9bf4000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:48.146 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.145+0000 7f9bcffff640 3 client.5260.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:48.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.145+0000 7f9bcffff640 3 client.5260.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:48.147 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.145+0000 7f9bcffff640 20 client.5260.objecter dump_active .. 0 homeless 2026-02-24T18:40:48.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.151+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9be4061fd0 con 0x7f9bf40c94b0 2026-02-24T18:40:48.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.151+0000 7f9bcffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.153 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.151+0000 7f9bcffff640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:48.235 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.234+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "10", "target": ["mon-mgr", ""]}) -- 0x7f9bf40c8240 con 0x7f9bc4078e50 2026-02-24T18:40:48.236 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.234+0000 7f9bcffff640 1 -- 10.20.193.18:0/3001434126 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 10 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f9bf40c8240 con 0x7f9bc4078e50 2026-02-24T18:40:48.236 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 10 to deep-scrub 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 10 librados: watch_flush enter 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 10 librados: watch_flush exit 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 20 client.5260.objecter shutdown clearing up homeless session... 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 10 client.5260.objecter successfully canceled tick 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 msgr2=0x7f9bc407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 0x7f9bc407b310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0004660 tx=0x7f9be0009210 comp rx=0 tx=0).stop 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 10 monclient: shutdown 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.238 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 msgr2=0x7f9bf41581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 0x7f9bf41581b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9be4006dc0 tx=0x7f9be400ed70 comp rx=0 tx=0).stop 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 shutdown_connections 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f9bc4078e50 0x7f9bc407b310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f9bf40c94b0 0x7f9bf41581b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f9bf4158e90 0x7f9bf414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 --2- 10.20.193.18:0/3001434126 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f9bf40c90d0 0x7f9bf4148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 >> 10.20.193.18:0/3001434126 conn(0x7f9bf40cd780 msgr2=0x7f9bf414b980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 shutdown_connections 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 -- 10.20.193.18:0/3001434126 wait complete. 2026-02-24T18:40:48.239 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.235+0000 7f9bf9f6f640 1 librados: shutdown 2026-02-24T18:40:48.246 INFO:tasks.ceph:Scrubbing osd.14 2026-02-24T18:40:48.246 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:48.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.314+0000 7fe9214c9640 10 monclient: get_monmap_and_config 2026-02-24T18:40:48.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.314+0000 7fe9214c9640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.316 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.314+0000 7fe9214c9640 10 monclient: monmap: 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:48.315803+0000 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:48.315803+0000 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:48.317 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:48.318 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 Processor -- start 2026-02-24T18:40:48.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 -- start start 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient: init 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient: _add_conns ranks=[4,3,1,2,0,5] 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe91c1503e0 0x7fe91c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe91c1503e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe91c15a070 0x7fe91c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe91c15a070 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe91c0702f0 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe91c15a570 con 0x7fe91c0702f0 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7fe91c0c7d70 con 0x7fe91c15a070 2026-02-24T18:40:48.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe91c0c8610 con 0x7fe91c1503e0 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.315+0000 7fe9214c9640 10 monclient(hunting): authenticate will time out at 1791.996975s 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe91c1503e0 0x7fe91c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91a7fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7fe91c15a070 0x7fe91c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7fe91c1503e0 0x7fe91c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52412/0 (socket says 10.20.193.18:52412) 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 1 -- 10.20.193.18:0/3688534461 learned_addr learned my addr 10.20.193.18:0/3688534461 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 10 monclient(hunting): get_auth_request con 0x7fe91c1503e0 auth_method 0 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91a7fc640 10 monclient(hunting): get_auth_request con 0x7fe91c15a070 auth_method 0 2026-02-24T18:40:48.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.316+0000 7fe91a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 10 monclient(hunting): handle_auth_done global_id 5263 payload 274 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 1 -- 10.20.193.18:0/3688534461 >> v2:10.20.193.18:3300/0 conn(0x7fe91c15a070 msgr2=0x7fe91c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 1 --2- 10.20.193.18:0/3688534461 >> v2:10.20.193.18:3300/0 conn(0x7fe91c15a070 0x7fe91c06fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.317+0000 7fe91affd640 1 -- 10.20.193.18:0/3688534461 >> v1:10.20.193.32:6789/0 conn(0x7fe91c0702f0 legacy=0x7fe91c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.318+0000 7fe91affd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.318+0000 7fe91affd640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.318+0000 7fe91affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.318+0000 7fe91affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.318+0000 7fe91affd640 1 -- 10.20.193.18:0/3688534461 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c15a850 con 0x7fe91c1503e0 2026-02-24T18:40:48.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.319+0000 7fe91affd640 1 --2- 10.20.193.18:0/3688534461 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c1503e0 0x7fe91c159b30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0c5960 tx=0x7fe91002ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f90883975b89d3 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 1 -- 10.20.193.18:0/3688534461 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91003c070 con 0x7fe91c1503e0 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: dump: 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.331 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 1 -- 10.20.193.18:0/3688534461 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91002fba0 con 0x7fe91c1503e0 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 5 monclient: authenticate success, global_id 5263 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 1 -- 10.20.193.18:0/3688534461 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910037440 con 0x7fe91c1503e0 2026-02-24T18:40:48.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 4 set_mon_vals no callback set 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe919ffb640 10 monclient: dump: 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.333 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 10 monclient: shutdown 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 -- 10.20.193.18:0/3688534461 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c1503e0 msgr2=0x7fe91c159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 --2- 10.20.193.18:0/3688534461 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c1503e0 0x7fe91c159b30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0c5960 tx=0x7fe91002ef70 comp rx=0 tx=0).stop 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 -- 10.20.193.18:0/3688534461 shutdown_connections 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 --2- 10.20.193.18:0/3688534461 >> v2:10.20.193.18:3300/0 conn(0x7fe91c15a070 0x7fe91c06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 --2- 10.20.193.18:0/3688534461 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c1503e0 0x7fe91c159b30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 -- 10.20.193.18:0/3688534461 >> 10.20.193.18:0/3688534461 conn(0x7fe91c0cd600 msgr2=0x7fe91c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 -- 10.20.193.18:0/3688534461 shutdown_connections 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.321+0000 7fe9214c9640 1 -- 10.20.193.18:0/3688534461 wait complete. 2026-02-24T18:40:48.334 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient: monmap: 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 librados: starting msgr at 2026-02-24T18:40:48.335 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 librados: starting objecter 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 Processor -- start 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 -- start start 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 librados: setting wanted keys 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 librados: calling monclient init 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient: init 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c0702f0 0x7fe91c1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-b con 0x7fe91c0702f0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe91c1503e0 0x7fe91c1f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-a con 0x7fe91c1503e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe91c15a070 0x7fe91c0c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.336 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): picked mon.noname-c con 0x7fe91c15a070 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c0c82a0 con 0x7fe91c1503e0 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c070760 con 0x7fe91c0702f0 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1507c0 con 0x7fe91c15a070 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.322+0000 7fe9214c9640 10 monclient(hunting): authenticate will time out at 1792.003781s 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe91c1503e0 0x7fe91c1f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c0702f0 0x7fe91c1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe91c15a070 0x7fe91c0c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.337 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe91c1503e0 0x7fe91c1f15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49620/0 (socket says 10.20.193.18:49620) 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 1 -- 10.20.193.18:0/3884726913 learned_addr learned my addr 10.20.193.18:0/3884726913 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): get_auth_request con 0x7fe91c0702f0 auth_method 0 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): get_auth_request con 0x7fe91c1503e0 auth_method 0 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): get_auth_request con 0x7fe91c15a070 auth_method 0 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.338 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.323+0000 7fe91b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 10 monclient(hunting): handle_auth_done global_id 5266 payload 824 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 -- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe91c15a070 msgr2=0x7fe91c0c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fe91c15a070 0x7fe91c0c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 -- 10.20.193.18:0/3884726913 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe91c1503e0 msgr2=0x7fe91c1f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.339 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fe91c1503e0 0x7fe91c1f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.324+0000 7fe91affd640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c1f1d50 con 0x7fe91c0702f0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.325+0000 7fe91affd640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c0702f0 0x7fe91c1f5590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0c7ef0 tx=0x7fe9100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.325+0000 7fe8fb7fe640 10 client.?.objecter ms_handle_connect 0x7fe91c0702f0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.325+0000 7fe8fb7fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910047070 con 0x7fe91c0702f0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.340 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: dump: 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe910042450 con 0x7fe91c0702f0 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 5 monclient: authenticate success, global_id 5266 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 10 monclient: _renew_subs 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c1f2030 con 0x7fe91c0702f0 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91003c040 con 0x7fe91c0702f0 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe8fb7fe640 10 monclient: dump: 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.342 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9197fa640 4 set_mon_vals no callback set 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 10 client.5266.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 10 monclient: _renew_subs 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c0c1380 con 0x7fe91c0702f0 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.327+0000 7fe9214c9640 1 librados: init done 2026-02-24T18:40:48.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 10 client.5266.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 15 client.5266.objecter _session_command_op_assign -1 1 2026-02-24T18:40:48.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 20 client.5266.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:48.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 10 client.5266.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.344 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.328+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe91c159b30 con 0x7fe91c0702f0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.339+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe91004b3b0 con 0x7fe91c0702f0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.339+0000 7fe8fb7fe640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe8e8078e50 0x7fe8e807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.339+0000 7fe91a7fc640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe8e8078e50 0x7fe8e807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.339+0000 7fe91a7fc640 10 monclient: get_auth_request con 0x7fe8e8078e50 auth_method 0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.340+0000 7fe8fb7fe640 10 client.5266.objecter ms_handle_connect 0x7fe8e8078e50 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.340+0000 7fe91a7fc640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe8e8078e50 0x7fe8e807b310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe91c1549a0 tx=0x7fe904009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fe910082b00 con 0x7fe91c0702f0 2026-02-24T18:40:48.346 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 10 client.5266.objecter ms_dispatch2 0x7fe91c000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 3 client.5266.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 3 client.5266.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 10 client.5266.objecter checking command 1 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7fe8e8081330 0x7fe8e8083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter _get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 15 client.5266.objecter _session_command_op_remove -1 1 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter _get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 15 client.5266.objecter _session_command_op_assign 14 1 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 4 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 10 client.5266.objecter _send_command 1 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe91002fba0 con 0x7fe8e8081330 2026-02-24T18:40:48.347 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 20 client.5266.objecter dump_active .. 0 homeless 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe910082ec0 con 0x7fe91c0702f0 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.342+0000 7fe8fb7fe640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.343+0000 7fe91b7fe640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7fe8e8081330 0x7fe8e8083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.343+0000 7fe91b7fe640 10 monclient: get_auth_request con 0x7fe8e8081330 auth_method 0 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.345+0000 7fe91b7fe640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7fe8e8081330 0x7fe8e8083750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.348 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.345+0000 7fe8fb7fe640 10 client.5266.objecter ms_handle_connect 0x7fe8e8081330 2026-02-24T18:40:48.356 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.355+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== osd.14 v2:10.20.193.110:6816/3577843273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fe91002fba0 con 0x7fe8e8081330 2026-02-24T18:40:48.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.355+0000 7fe8fb7fe640 10 client.5266.objecter ms_dispatch2 0x7fe91c000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:48.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.355+0000 7fe8fb7fe640 10 client.5266.objecter _finish_command 1 = system:0 2026-02-24T18:40:48.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.355+0000 7fe8fb7fe640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 4 2026-02-24T18:40:48.357 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.355+0000 7fe8fb7fe640 15 client.5266.objecter _session_command_op_remove 14 1 2026-02-24T18:40:48.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 10 client.5266.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:48.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 15 client.5266.objecter _session_command_op_assign -1 2 2026-02-24T18:40:48.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 20 client.5266.objecter _get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.363 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 20 client.5266.objecter _get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 15 client.5266.objecter _session_command_op_remove -1 2 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 20 client.5266.objecter get_session s=0x7fe8e8080c60 osd=14 3 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 15 client.5266.objecter _session_command_op_assign 14 2 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 4 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 10 client.5266.objecter _send_command 2 2026-02-24T18:40:48.364 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.361+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 --> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe91c14fc00 con 0x7fe8e8081330 2026-02-24T18:40:48.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8fb7fe640 1 -- 10.20.193.18:0/3884726913 <== osd.14 v2:10.20.193.110:6816/3577843273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe91c14fc00 con 0x7fe8e8081330 2026-02-24T18:40:48.370 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8fb7fe640 10 client.5266.objecter ms_dispatch2 0x7fe91c000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8fb7fe640 10 client.5266.objecter _finish_command 2 = system:0 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8fb7fe640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 4 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8fb7fe640 15 client.5266.objecter _session_command_op_remove 14 2 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:48.371 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 10 librados: watch_flush enter 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 10 librados: watch_flush exit 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 10 client.5266.objecter close_session for osd.14 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7fe8e8081330 msgr2=0x7fe8e8083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.110:6816/3577843273,v1:10.20.193.110:6817/3577843273] conn(0x7fe8e8081330 0x7fe8e8083750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 20 client.5266.objecter put_session s=0x7fe8e8080c60 osd=14 1 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 20 client.5266.objecter shutdown clearing up homeless session... 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 10 client.5266.objecter successfully canceled tick 2026-02-24T18:40:48.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe8e8078e50 msgr2=0x7fe8e807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fe8e8078e50 0x7fe8e807b310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe91c1549a0 tx=0x7fe904009290 comp rx=0 tx=0).stop 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 10 monclient: shutdown 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c0702f0 msgr2=0x7fe91c1f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 --2- 10.20.193.18:0/3884726913 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fe91c0702f0 0x7fe91c1f5590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0c7ef0 tx=0x7fe9100047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe91b7fe640 1 -- 10.20.193.18:0/3884726913 reap_dead start 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 shutdown_connections 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 >> 10.20.193.18:0/3884726913 conn(0x7fe91c0cd600 msgr2=0x7fe91c14efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 shutdown_connections 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 -- 10.20.193.18:0/3884726913 wait complete. 2026-02-24T18:40:48.374 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.369+0000 7fe8f97fa640 1 librados: shutdown 2026-02-24T18:40:48.380 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 10 monclient: get_monmap_and_config 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 10 monclient: monmap: 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:48.448346+0000 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:48.448346+0000 2026-02-24T18:40:48.449 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:48.450 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 1 Processor -- start 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 1 -- start start 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.447+0000 7fd1f0c7c640 10 monclient: init 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient: _add_conns ranks=[2,0,5,1,3,4] 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd1ec1527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd1ec152bb0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd1ec0c6190 0x7fd1ec0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-c con 0x7fd1ec0c6190 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ec06e300 con 0x7fd1ec152bb0 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ec06e5e0 con 0x7fd1ec1527d0 2026-02-24T18:40:48.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ec06f1f0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1f0c7c640 10 monclient(hunting): authenticate will time out at 1792.129490s 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1ead76640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7fd1ec0c6190 0x7fd1ec0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1e9d74640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7fd1ec152bb0 0x7fd1ec0c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48690/0 (socket says 10.20.193.18:48690) 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.448+0000 7fd1e9d74640 1 -- 10.20.193.18:0/763762443 learned_addr learned my addr 10.20.193.18:0/763762443 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): get_auth_request con 0x7fd1ec0c6190 auth_method 0 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.449+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 10 monclient(hunting): handle_auth_done global_id 5246 payload 274 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 1 -- 10.20.193.18:0/763762443 >> v1:10.20.193.110:6789/0 conn(0x7fd1ec1527d0 legacy=0x7fd1ec06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 1 -- 10.20.193.18:0/763762443 >> v1:10.20.193.18:6789/0 conn(0x7fd1ec152bb0 legacy=0x7fd1ec0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 1 monclient: found mon.noname-c 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1e9573640 1 -- 10.20.193.18:0/763762443 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550039883 0 0) 0x7fd1ec06e300 con 0x7fd1ec152bb0 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1ead76640 1 -- 10.20.193.18:0/763762443 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec06e8c0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.450+0000 7fd1e9573640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-24T18:40:48.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.451+0000 7fd1ead76640 1 --2- 10.20.193.18:0/763762443 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec0c8e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06f370 tx=0x7fd1dc02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3953b03e674a30 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 1 -- 10.20.193.18:0/763762443 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc03c070 con 0x7fd1ec0c6190 2026-02-24T18:40:48.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: dump: 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.461 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 1 -- 10.20.193.18:0/763762443 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc037440 con 0x7fd1ec0c6190 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 5 monclient: authenticate success, global_id 5246 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 1 -- 10.20.193.18:0/763762443 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc037810 con 0x7fd1ec0c6190 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 4 set_mon_vals no callback set 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:48.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1e9573640 10 monclient: dump: 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 10 monclient: shutdown 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/763762443 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec0c6190 msgr2=0x7fd1ec0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.453+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/763762443 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec0c8e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06f370 tx=0x7fd1dc02f1e0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/763762443 shutdown_connections 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/763762443 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec0c8e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/763762443 >> 10.20.193.18:0/763762443 conn(0x7fd1ec0cd780 msgr2=0x7fd1ec0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/763762443 shutdown_connections 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/763762443 wait complete. 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 10 monclient: monmap: 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:48.464 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 librados: starting msgr at 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 librados: starting objecter 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 Processor -- start 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 -- start start 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 librados: setting wanted keys 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 1 librados: calling monclient init 2026-02-24T18:40:48.465 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.454+0000 7fd1f0c7c640 10 monclient: init 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec1f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-b con 0x7fd1ec0c6190 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec1527d0 0x7fd1ec0c1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-c con 0x7fd1ec1527d0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 0x7fd1ec14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): picked mon.noname-a con 0x7fd1ec152bb0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec0c93a0 con 0x7fd1ec152bb0 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec06ee80 con 0x7fd1ec0c6190 2026-02-24T18:40:48.466 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec0c9520 con 0x7fd1ec1527d0 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1f0c7c640 10 monclient(hunting): authenticate will time out at 1792.136191s 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 0x7fd1ec14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ea575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec1f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 0x7fd1ec14db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49628/0 (socket says 10.20.193.18:49628) 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 1 -- 10.20.193.18:0/2721779795 learned_addr learned my addr 10.20.193.18:0/2721779795 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 10 monclient(hunting): get_auth_request con 0x7fd1ec152bb0 auth_method 0 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ead76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1e9d74640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec1527d0 0x7fd1ec0c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ea575640 10 monclient(hunting): get_auth_request con 0x7fd1ec0c6190 auth_method 0 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ea575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ea575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1ea575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1e9d74640 10 monclient(hunting): get_auth_request con 0x7fd1ec1527d0 auth_method 0 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1e9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1e9d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.455+0000 7fd1e9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ead76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ea575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ea575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1ea575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1e9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1e9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.456+0000 7fd1e9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 10 monclient(hunting): handle_auth_done global_id 5269 payload 824 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 -- 10.20.193.18:0/2721779795 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec1527d0 msgr2=0x7fd1ec0c1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec1527d0 0x7fd1ec0c1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 -- 10.20.193.18:0/2721779795 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 msgr2=0x7fd1ec14db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 0x7fd1ec14db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.457+0000 7fd1ea575640 1 -- 10.20.193.18:0/2721779795 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec14e520 con 0x7fd1ec0c6190 2026-02-24T18:40:48.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.458+0000 7fd1ea575640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec1f37e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b550 tx=0x7fd1e000ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.458+0000 7fd1d2ffd640 10 client.?.objecter ms_handle_connect 0x7fd1ec0c6190 2026-02-24T18:40:48.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.458+0000 7fd1d2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:48.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0016070 con 0x7fd1ec0c6190 2026-02-24T18:40:48.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: dump: 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.472 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e00044e0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 5 monclient: authenticate success, global_id 5269 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00029c0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1d2ffd640 10 monclient: dump: 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.473 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _renew_subs 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1e8d72640 4 set_mon_vals no callback set 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec14e800 con 0x7fd1ec0c6190 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 client.5269.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _renew_subs 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1f71d0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 1 librados: init done 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.460+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ec1f6ce0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.563+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd1e0002b60 con 0x7fd1ec0c6190 2026-02-24T18:40:48.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.564+0000 7fd1d2ffd640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 0x7fd1b807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.565 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.564+0000 7fd1e9d74640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 0x7fd1b807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.564+0000 7fd1e9d74640 10 monclient: get_auth_request con 0x7fd1b8078e00 auth_method 0 2026-02-24T18:40:48.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.565+0000 7fd1e9d74640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 0x7fd1b807b2c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0bd600 tx=0x7fd1d4007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.566 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.565+0000 7fd1d2ffd640 10 client.5269.objecter ms_handle_connect 0x7fd1b8078e00 2026-02-24T18:40:48.567 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.566+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fd1e00967b0 con 0x7fd1ec0c6190 2026-02-24T18:40:48.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.566+0000 7fd1d2ffd640 10 client.5269.objecter ms_dispatch2 0x7fd1ec000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:48.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.566+0000 7fd1d2ffd640 3 client.5269.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:48.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.566+0000 7fd1d2ffd640 3 client.5269.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:48.568 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.566+0000 7fd1d2ffd640 20 client.5269.objecter dump_active .. 0 homeless 2026-02-24T18:40:48.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.572+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd1e0060f00 con 0x7fd1ec0c6190 2026-02-24T18:40:48.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.572+0000 7fd1d2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.574 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.572+0000 7fd1d2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:48.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.656+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "14", "target": ["mon-mgr", ""]}) -- 0x7fd1ec0c5a60 con 0x7fd1b8078e00 2026-02-24T18:40:48.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.657+0000 7fd1d2ffd640 1 -- 10.20.193.18:0/2721779795 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 14 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fd1ec0c5a60 con 0x7fd1b8078e00 2026-02-24T18:40:48.658 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 14 to deep-scrub 2026-02-24T18:40:48.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.657+0000 7fd1f0c7c640 10 librados: watch_flush enter 2026-02-24T18:40:48.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 10 librados: watch_flush exit 2026-02-24T18:40:48.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 20 client.5269.objecter shutdown clearing up homeless session... 2026-02-24T18:40:48.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 10 client.5269.objecter successfully canceled tick 2026-02-24T18:40:48.660 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 msgr2=0x7fd1b807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 0x7fd1b807b2c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0bd600 tx=0x7fd1d4007c00 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 10 monclient: shutdown 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 msgr2=0x7fd1ec1f37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec1f37e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b550 tx=0x7fd1e000ba20 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 shutdown_connections 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fd1b8078e00 0x7fd1b807b2c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fd1ec152bb0 0x7fd1ec14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fd1ec1527d0 0x7fd1ec0c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 --2- 10.20.193.18:0/2721779795 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fd1ec0c6190 0x7fd1ec1f37e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 >> 10.20.193.18:0/2721779795 conn(0x7fd1ec0cd780 msgr2=0x7fd1ec0c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.661 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 shutdown_connections 2026-02-24T18:40:48.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 -- 10.20.193.18:0/2721779795 wait complete. 2026-02-24T18:40:48.662 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.658+0000 7fd1f0c7c640 1 librados: shutdown 2026-02-24T18:40:48.668 INFO:tasks.ceph:Scrubbing osd.3 2026-02-24T18:40:48.668 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.737+0000 7efe6cfb3640 10 monclient: get_monmap_and_config 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.737+0000 7efe6cfb3640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.737+0000 7efe6cfb3640 10 monclient: monmap: 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:48.738404+0000 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:48.738404+0000 2026-02-24T18:40:48.739 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:48.740 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.737+0000 7efe6cfb3640 1 Processor -- start 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 -- start start 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient: init 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient: _add_conns ranks=[4,2,5,3,0,1] 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efe681503e0 0x7efe68159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-b con 0x7efe681503e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efe6815a070 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:48.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7efe680704c0 0x7efe680c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-c con 0x7efe680704c0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6815a570 con 0x7efe6815a070 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7efe680c7d70 con 0x7efe681503e0 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7efe680c8610 con 0x7efe680704c0 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.738+0000 7efe6cfb3640 10 monclient(hunting): authenticate will time out at 1792.419544s 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efe681503e0 0x7efe68159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7efe680704c0 0x7efe680c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7efe681503e0 0x7efe68159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52444/0 (socket says 10.20.193.18:52444) 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 1 -- 10.20.193.18:0/2171337712 learned_addr learned my addr 10.20.193.18:0/2171337712 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): get_auth_request con 0x7efe681503e0 auth_method 0 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): get_auth_request con 0x7efe680704c0 auth_method 0 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.739+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 10 monclient(hunting): handle_auth_done global_id 5272 payload 274 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 1 -- 10.20.193.18:0/2171337712 >> v2:10.20.193.110:3300/0 conn(0x7efe680704c0 msgr2=0x7efe680c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 1 --2- 10.20.193.18:0/2171337712 >> v2:10.20.193.110:3300/0 conn(0x7efe680704c0 0x7efe680c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 1 -- 10.20.193.18:0/2171337712 >> v1:10.20.193.110:6789/0 conn(0x7efe6815a070 legacy=0x7efe6806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.740+0000 7efe66575640 1 -- 10.20.193.18:0/2171337712 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6815a850 con 0x7efe681503e0 2026-02-24T18:40:48.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.741+0000 7efe66575640 1 --2- 10.20.193.18:0/2171337712 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe681503e0 0x7efe68159b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe6806f550 tx=0x7efe5c02ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30fed6f691d7598 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 1 -- 10.20.193.18:0/2171337712 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c03c070 con 0x7efe681503e0 2026-02-24T18:40:48.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: dump: 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 5 monclient: authenticate success, global_id 5272 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 1 -- 10.20.193.18:0/2171337712 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c02fbd0 con 0x7efe681503e0 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 1 -- 10.20.193.18:0/2171337712 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c037440 con 0x7efe681503e0 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 4 set_mon_vals no callback set 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe65573640 10 monclient: dump: 2026-02-24T18:40:48.755 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 10 monclient: shutdown 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2171337712 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe681503e0 msgr2=0x7efe68159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2171337712 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe681503e0 0x7efe68159b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe6806f550 tx=0x7efe5c02ef60 comp rx=0 tx=0).stop 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2171337712 shutdown_connections 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2171337712 >> v2:10.20.193.110:3300/0 conn(0x7efe680704c0 0x7efe680c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2171337712 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe681503e0 0x7efe68159b30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2171337712 >> 10.20.193.18:0/2171337712 conn(0x7efe680cd600 msgr2=0x7efe680cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2171337712 shutdown_connections 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2171337712 wait complete. 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 10 monclient: monmap: 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.757 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 librados: starting msgr at 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.744+0000 7efe6cfb3640 1 librados: starting objecter 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 Processor -- start 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 -- start start 2026-02-24T18:40:48.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 librados: setting wanted keys 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 librados: calling monclient init 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient: init 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 0x7efe68149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-b con 0x7efe680704c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efe681503e0 0x7efe681f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-c con 0x7efe681503e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efe6815a070 0x7efe6814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): picked mon.noname-a con 0x7efe6815a070 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efe681507c0 con 0x7efe6815a070 2026-02-24T18:40:48.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe680c82a0 con 0x7efe680704c0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe68150940 con 0x7efe681503e0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe6cfb3640 10 monclient(hunting): authenticate will time out at 1792.426471s 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe65d74640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efe681503e0 0x7efe681f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe66575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 0x7efe68149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.745+0000 7efe66d76640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efe6815a070 0x7efe6814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 0x7efe68149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:52460/0 (socket says 10.20.193.18:52460) 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 1 -- 10.20.193.18:0/2535639205 learned_addr learned my addr 10.20.193.18:0/2535639205 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): get_auth_request con 0x7efe680704c0 auth_method 0 2026-02-24T18:40:48.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): get_auth_request con 0x7efe681503e0 auth_method 0 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): get_auth_request con 0x7efe6815a070 auth_method 0 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe65d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.746+0000 7efe66d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 10 monclient(hunting): handle_auth_done global_id 5275 payload 824 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 -- 10.20.193.18:0/2535639205 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efe681503e0 msgr2=0x7efe681f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efe681503e0 0x7efe681f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 -- 10.20.193.18:0/2535639205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efe6815a070 msgr2=0x7efe6814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efe6815a070 0x7efe6814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 monclient: found mon.noname-b 2026-02-24T18:40:48.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.747+0000 7efe66575640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681553c0 con 0x7efe680704c0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.748+0000 7efe66575640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 0x7efe68149310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe5c002410 tx=0x7efe5c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.748+0000 7efe4effd640 10 client.?.objecter ms_handle_connect 0x7efe680704c0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.748+0000 7efe4effd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c044070 con 0x7efe680704c0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: dump: 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.763 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 5 monclient: authenticate success, global_id 5275 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c040410 con 0x7efe680704c0 2026-02-24T18:40:48.764 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c03c040 con 0x7efe680704c0 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 monclient: _renew_subs 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681550e0 con 0x7efe680704c0 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe64d72640 4 set_mon_vals no callback set 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe4effd640 10 monclient: dump: 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.765 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 client.5275.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 monclient: _renew_subs 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe68149ef0 con 0x7efe680704c0 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 1 librados: init done 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 client.5275.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 15 client.5275.objecter _session_command_op_assign -1 1 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 20 client.5275.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:48.766 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 client.5275.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:48.767 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.750+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe680c5d90 con 0x7efe680704c0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.762+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efe5c03f050 con 0x7efe680704c0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.762+0000 7efe4effd640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe34078e50 0x7efe3407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.762+0000 7efe65d74640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe34078e50 0x7efe3407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.762+0000 7efe65d74640 10 monclient: get_auth_request con 0x7efe34078e50 auth_method 0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.763+0000 7efe65d74640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe34078e50 0x7efe3407b310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe680cd210 tx=0x7efe50009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.763+0000 7efe4effd640 10 client.5275.objecter ms_handle_connect 0x7efe34078e50 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7efe5c0bc570 con 0x7efe680704c0 2026-02-24T18:40:48.769 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 10 client.5275.objecter ms_dispatch2 0x7efe68000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 3 client.5275.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 3 client.5275.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 10 client.5275.objecter checking command 1 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7efe34081330 0x7efe34083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter _get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 15 client.5275.objecter _session_command_op_remove -1 1 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter _get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 15 client.5275.objecter _session_command_op_assign 3 1 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 4 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 10 client.5275.objecter _send_command 1 2026-02-24T18:40:48.770 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe5c033ec0 con 0x7efe34081330 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.764+0000 7efe4effd640 20 client.5275.objecter dump_active .. 0 homeless 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.765+0000 7efe66d76640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7efe34081330 0x7efe34083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.765+0000 7efe66d76640 10 monclient: get_auth_request con 0x7efe34081330 auth_method 0 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.765+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe5c083250 con 0x7efe680704c0 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.765+0000 7efe4effd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.767+0000 7efe66d76640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7efe34081330 0x7efe34083750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.767+0000 7efe4effd640 10 client.5275.objecter ms_handle_connect 0x7efe34081330 2026-02-24T18:40:48.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.776+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== osd.3 v2:10.20.193.112:6826/230320724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7efe5c033ec0 con 0x7efe34081330 2026-02-24T18:40:48.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.776+0000 7efe4effd640 10 client.5275.objecter ms_dispatch2 0x7efe68000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:48.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.776+0000 7efe4effd640 10 client.5275.objecter _finish_command 1 = system:0 2026-02-24T18:40:48.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.776+0000 7efe4effd640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 4 2026-02-24T18:40:48.778 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.776+0000 7efe4effd640 15 client.5275.objecter _session_command_op_remove 3 1 2026-02-24T18:40:48.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 10 client.5275.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:48.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 15 client.5275.objecter _session_command_op_assign -1 2 2026-02-24T18:40:48.784 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 20 client.5275.objecter _get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 20 client.5275.objecter _get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 15 client.5275.objecter _session_command_op_remove -1 2 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 20 client.5275.objecter get_session s=0x7efe34080c60 osd=3 3 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 15 client.5275.objecter _session_command_op_assign 3 2 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 4 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 10 client.5275.objecter _send_command 2 2026-02-24T18:40:48.785 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.782+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 --> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efe6815e200 con 0x7efe34081330 2026-02-24T18:40:48.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe4effd640 1 -- 10.20.193.18:0/2535639205 <== osd.3 v2:10.20.193.112:6826/230320724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7efe6815e200 con 0x7efe34081330 2026-02-24T18:40:48.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe4effd640 10 client.5275.objecter ms_dispatch2 0x7efe68000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:48.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe4effd640 10 client.5275.objecter _finish_command 2 = system:0 2026-02-24T18:40:48.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe4effd640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 4 2026-02-24T18:40:48.798 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe4effd640 15 client.5275.objecter _session_command_op_remove 3 2 2026-02-24T18:40:48.799 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:48.799 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:48.799 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:48.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 10 librados: watch_flush enter 2026-02-24T18:40:48.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 10 librados: watch_flush exit 2026-02-24T18:40:48.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 10 client.5275.objecter close_session for osd.3 2026-02-24T18:40:48.800 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7efe34081330 msgr2=0x7efe34083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6826/230320724,v1:10.20.193.112:6827/230320724] conn(0x7efe34081330 0x7efe34083750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 20 client.5275.objecter put_session s=0x7efe34080c60 osd=3 1 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 20 client.5275.objecter shutdown clearing up homeless session... 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 10 client.5275.objecter successfully canceled tick 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe34078e50 msgr2=0x7efe3407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efe34078e50 0x7efe3407b310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe680cd210 tx=0x7efe50009110 comp rx=0 tx=0).stop 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 10 monclient: shutdown 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 msgr2=0x7efe68149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 --2- 10.20.193.18:0/2535639205 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efe680704c0 0x7efe68149310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe5c002410 tx=0x7efe5c0047c0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe66d76640 1 -- 10.20.193.18:0/2535639205 reap_dead start 2026-02-24T18:40:48.801 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 shutdown_connections 2026-02-24T18:40:48.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 >> 10.20.193.18:0/2535639205 conn(0x7efe680cd600 msgr2=0x7efe680c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 shutdown_connections 2026-02-24T18:40:48.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 -- 10.20.193.18:0/2535639205 wait complete. 2026-02-24T18:40:48.802 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.797+0000 7efe6cfb3640 1 librados: shutdown 2026-02-24T18:40:48.807 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-24T18:40:48.874 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 10 monclient: get_monmap_and_config 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 10 monclient: monmap: 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:48.874208+0000 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:48.874208+0000 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.875 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:48.876 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 1 Processor -- start 2026-02-24T18:40:48.878 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 1 -- start start 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.873+0000 7f581c308640 10 monclient: init 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient: _add_conns ranks=[2,5,0,1,4,3] 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f581406ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f58140c99a0 0x7f5814070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): picked mon.noname-c con 0x7f58140c99a0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f581415a5e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140c7e90 con 0x7f581415a5e0 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140c8170 con 0x7f581406ff20 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5814159a40 con 0x7f58140c99a0 2026-02-24T18:40:48.879 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581c308640 10 monclient(hunting): authenticate will time out at 1792.555457s 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581987c640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f58140c99a0 0x7f5814070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581a87e640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f581415a5e0 0x7f581414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:48706/0 (socket says 10.20.193.18:48706) 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.874+0000 7f581a87e640 1 -- 10.20.193.18:0/1242625202 learned_addr learned my addr 10.20.193.18:0/1242625202 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): get_auth_request con 0x7f58140c99a0 auth_method 0 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.875+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.880 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 1 -- 10.20.193.18:0/1242625202 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991018415 0 0) 0x7f58140c7e90 con 0x7f581415a5e0 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 10 monclient(hunting): _negotiate 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 10 monclient(hunting): my global_id is 5349 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581907b640 1 -- 10.20.193.18:0/1242625202 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57fc003620 con 0x7f581415a5e0 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 10 monclient(hunting): handle_auth_done global_id 5258 payload 274 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 1 -- 10.20.193.18:0/1242625202 >> v1:10.20.193.110:6789/0 conn(0x7f581406ff20 legacy=0x7f5814159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 1 -- 10.20.193.18:0/1242625202 >> v1:10.20.193.18:6789/0 conn(0x7f581415a5e0 legacy=0x7f581414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 1 monclient: found mon.noname-c 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.881 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.876+0000 7f581987c640 1 -- 10.20.193.18:0/1242625202 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140c8450 con 0x7f58140c99a0 2026-02-24T18:40:48.886 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.877+0000 7f581987c640 1 --2- 10.20.193.18:0/1242625202 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f58140c99a0 0x7f5814070300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58080098d0 tx=0x7f580802ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2557de224212456e server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 1 -- 10.20.193.18:0/1242625202 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580803c070 con 0x7f58140c99a0 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: dump: 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.887 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 1 -- 10.20.193.18:0/1242625202 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580802fb00 con 0x7f58140c99a0 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 1 -- 10.20.193.18:0/1242625202 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808037440 con 0x7f58140c99a0 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 5 monclient: authenticate success, global_id 5258 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 4 set_mon_vals no callback set 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.888 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581907b640 10 monclient: dump: 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.889 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 10 monclient: shutdown 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 1 -- 10.20.193.18:0/1242625202 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f58140c99a0 msgr2=0x7f5814070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.880+0000 7f581c308640 1 --2- 10.20.193.18:0/1242625202 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f58140c99a0 0x7f5814070300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58080098d0 tx=0x7f580802ee80 comp rx=0 tx=0).stop 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 -- 10.20.193.18:0/1242625202 shutdown_connections 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 --2- 10.20.193.18:0/1242625202 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f58140c99a0 0x7f5814070300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 -- 10.20.193.18:0/1242625202 >> 10.20.193.18:0/1242625202 conn(0x7f58140cd780 msgr2=0x7f58140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 -- 10.20.193.18:0/1242625202 shutdown_connections 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 -- 10.20.193.18:0/1242625202 wait complete. 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 10 monclient: build_initial_monmap 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 10 monclient: monmap: 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:48.890 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 librados: starting msgr at 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 librados: starting objecter 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 Processor -- start 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 -- start start 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 librados: setting wanted keys 2026-02-24T18:40:48.891 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 1 librados: calling monclient init 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.881+0000 7f581c308640 10 monclient: init 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 0x7f5814149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): picked mon.noname-c con 0x7f581406ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 0x7f58141f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): picked mon.noname-a con 0x7f58140c99a0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f581415a5e0 0x7f58141f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): picked mon.noname-b con 0x7f581415a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5814150520 con 0x7f58140c99a0 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.892 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58140c8a10 con 0x7f581415a5e0 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58141506a0 con 0x7f581406ff20 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581c308640 10 monclient(hunting): authenticate will time out at 1792.563236s 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 0x7f58141f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 0x7f5814149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a87e640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f581415a5e0 0x7f58141f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 0x7f58141f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:49644/0 (socket says 10.20.193.18:49644) 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 1 -- 10.20.193.18:0/1698226666 learned_addr learned my addr 10.20.193.18:0/1698226666 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): get_auth_request con 0x7f58140c99a0 auth_method 0 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.893 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): get_auth_request con 0x7f581406ff20 auth_method 0 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a87e640 10 monclient(hunting): get_auth_request con 0x7f581415a5e0 auth_method 0 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a87e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a87e640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a87e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581987c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.894 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.882+0000 7f581a07d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.883+0000 7f581a87e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.883+0000 7f581a87e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.883+0000 7f581a87e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 10 monclient(hunting): handle_auth_done global_id 5261 payload 824 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 -- 10.20.193.18:0/1698226666 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f581415a5e0 msgr2=0x7f58141f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f581415a5e0 0x7f58141f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 -- 10.20.193.18:0/1698226666 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 msgr2=0x7f58141f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 0x7f58141f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 10 monclient: _finish_hunting 0 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 monclient: found mon.noname-c 2026-02-24T18:40:48.895 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:48.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.884+0000 7f581a07d640 1 -- 10.20.193.18:0/1698226666 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141f71d0 con 0x7f581406ff20 2026-02-24T18:40:48.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.885+0000 7f581a07d640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 0x7f5814149c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f580400b550 tx=0x7f580400ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.885+0000 7f57faffd640 10 client.?.objecter ms_handle_connect 0x7f581406ff20 2026-02-24T18:40:48.896 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.885+0000 7f57faffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004500 con 0x7f581406ff20 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: dump: 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.898 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 5 monclient: authenticate success, global_id 5261 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 10 monclient: _renew_subs 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141f74b0 con 0x7f581406ff20 2026-02-24T18:40:48.899 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 10 client.5261.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 10 monclient: _renew_subs 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141f7a70 con 0x7f581406ff20 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581c308640 1 librados: init done 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804015070 con 0x7f581406ff20 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804002980 con 0x7f581406ff20 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f581887a640 4 set_mon_vals no callback set 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.886+0000 7f57faffd640 10 monclient: dump: 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:48.900 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.887+0000 7f581c308640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.887+0000 7f581c308640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.887+0000 7f581c308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:48.901 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.887+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814070300 con 0x7f581406ff20 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.898+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5804002b20 con 0x7f581406ff20 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.898+0000 7f57faffd640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 0x7f57e407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.899+0000 7f581987c640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 0x7f57e407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.899+0000 7f581987c640 10 monclient: get_auth_request con 0x7f57e4078e00 auth_method 0 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.899+0000 7f57faffd640 10 client.5261.objecter ms_handle_connect 0x7f57e4078e00 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.899+0000 7f581987c640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 0x7f57e407b2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f580802eec0 tx=0x7f5808033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.901+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f5804097b30 con 0x7f581406ff20 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.901+0000 7f57faffd640 10 client.5261.objecter ms_dispatch2 0x7f5814000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.901+0000 7f57faffd640 3 client.5261.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.901+0000 7f57faffd640 3 client.5261.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:48.903 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:48.901+0000 7f57faffd640 20 client.5261.objecter dump_active .. 0 homeless 2026-02-24T18:40:49.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.000+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5804062280 con 0x7f581406ff20 2026-02-24T18:40:49.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.000+0000 7f57faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.001 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.000+0000 7f57faffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:49.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.081+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f581414f5b0 con 0x7f57e4078e00 2026-02-24T18:40:49.083 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.082+0000 7f57faffd640 1 -- 10.20.193.18:0/1698226666 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f581414f5b0 con 0x7f57e4078e00 2026-02-24T18:40:49.083 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 3 to deep-scrub 2026-02-24T18:40:49.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.082+0000 7f581c308640 10 librados: watch_flush enter 2026-02-24T18:40:49.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 10 librados: watch_flush exit 2026-02-24T18:40:49.085 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 20 client.5261.objecter shutdown clearing up homeless session... 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 10 client.5261.objecter successfully canceled tick 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 msgr2=0x7f57e407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 0x7f57e407b2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f580802eec0 tx=0x7f5808033000 comp rx=0 tx=0).stop 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 10 monclient: shutdown 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 msgr2=0x7f5814149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 0x7f5814149c70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f580400b550 tx=0x7f580400ba20 comp rx=0 tx=0).stop 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 shutdown_connections 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f57e4078e00 0x7f57e407b2c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f581415a5e0 0x7f58141f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f58140c99a0 0x7f58141f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.086 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 --2- 10.20.193.18:0/1698226666 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f581406ff20 0x7f5814149c70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 >> 10.20.193.18:0/1698226666 conn(0x7f58140cd780 msgr2=0x7f58140c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 shutdown_connections 2026-02-24T18:40:49.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 -- 10.20.193.18:0/1698226666 wait complete. 2026-02-24T18:40:49.087 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.083+0000 7f581c308640 1 librados: shutdown 2026-02-24T18:40:49.092 INFO:tasks.ceph:Scrubbing osd.7 2026-02-24T18:40:49.093 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:49.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 10 monclient: get_monmap_and_config 2026-02-24T18:40:49.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 10 monclient: monmap: 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:49.162926+0000 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:49.162926+0000 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:49.164 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:49.165 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:49.165 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 1 Processor -- start 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 1 -- start start 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.162+0000 7fa907914640 10 monclient: init 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient: _add_conns ranks=[2,1,0,5,4,3] 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa90015f680 addr v1:10.20.193.110:6789/0 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa90015fa60 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9000cd2a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90015da20 con 0x7fa9000cd2a0 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90015dce0 con 0x7fa90015fa60 2026-02-24T18:40:49.167 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90015dfc0 con 0x7fa90015f680 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa907914640 10 monclient(hunting): authenticate will time out at 1792.844215s 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa905689640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7fa90015f680 0x7fa900073210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:37220/0 (socket says 10.20.193.18:37220) 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.163+0000 7fa905689640 1 -- 10.20.193.18:0/3178535174 learned_addr learned my addr 10.20.193.18:0/3178535174 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.164+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39091243 0 0) 0x7fa90015dfc0 con 0x7fa90015f680 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.164+0000 7fa8effff640 10 monclient(hunting): _negotiate 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.164+0000 7fa8effff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.164+0000 7fa8effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.164+0000 7fa8effff640 10 monclient(hunting): my global_id is 5264 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8e8003620 con 0x7fa90015f680 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576302972 0 0) 0x7fa90015da20 con 0x7fa9000cd2a0 2026-02-24T18:40:49.168 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _negotiate 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): my global_id is 5355 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa90015dfc0 con 0x7fa9000cd2a0 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703566295 0 0) 0x7fa90015dce0 con 0x7fa90015fa60 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _negotiate 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 10 monclient(hunting): my global_id is 5281 2026-02-24T18:40:49.169 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.165+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa90015da20 con 0x7fa90015fa60 2026-02-24T18:40:49.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3975412098 0 0) 0x7fa8e8003620 con 0x7fa90015f680 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 >> v1:10.20.193.32:6789/0 conn(0x7fa90015fa60 legacy=0x7fa90015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 >> v1:10.20.193.18:6789/0 conn(0x7fa9000cd2a0 legacy=0x7fa9000cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90015e2a0 con 0x7fa90015f680 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7fa8f0002f00 con 0x7fa90015f680 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: dump: 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.178 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 >> v1:10.20.193.110:6789/0 conn(0x7fa90015f680 legacy=0x7fa900073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa8e8005010 0x7fa8e80038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): picked mon.c con 0x7fa8e8005010 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 0x7fa8e8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): picked mon.a con 0x7fa8e8005520 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa8e8009ab0 0x7fa8e800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): picked mon.b con 0x7fa8e8009ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e800c510 con 0x7fa8e8005520 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e800c6c0 con 0x7fa8e8009ab0 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e800c8c0 con 0x7fa8e8005010 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.166+0000 7fa8effff640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa904e88640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 0x7fa8e8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa905e8a640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa8e8009ab0 0x7fa8e800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa904e88640 10 monclient(hunting): get_auth_request con 0x7fa8e8005520 auth_method 0 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa904e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa904e88640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa904e88640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:49.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.167+0000 7fa905689640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa8e8005010 0x7fa8e80038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): get_auth_request con 0x7fa8e8009ab0 auth_method 0 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): get_auth_request con 0x7fa8e8005010 auth_method 0 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.168+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 10 monclient(hunting): handle_auth_done global_id 5264 payload 293 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 -- 10.20.193.18:0/3178535174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa8e8005010 msgr2=0x7fa8e80038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa8e8005010 0x7fa8e80038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 -- 10.20.193.18:0/3178535174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa8e8009ab0 msgr2=0x7fa8e800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa8e8009ab0 0x7fa8e800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 monclient: found mon.a 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa905e8a640 1 -- 10.20.193.18:0/3178535174 reap_dead start 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 -- 10.20.193.18:0/3178535174 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e8003620 con 0x7fa8e8005520 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa907914640 5 monclient: authenticate success, global_id 5264 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa907914640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.169+0000 7fa904e88640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 0x7fa8e8003e80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40031a0 tx=0x7fa8f402ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa29c5be906b0c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4002dc0 con 0x7fa8e8005520 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: dump: 2026-02-24T18:40:49.183 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f40052d0 con 0x7fa8e8005520 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa907914640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 1 -- 10.20.193.18:0/3178535174 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40036f0 con 0x7fa8e8005520 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa907914640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa907914640 4 set_mon_vals no callback set 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.171+0000 7fa8effff640 10 monclient: dump: 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.185 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 10 monclient: shutdown 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 -- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 msgr2=0x7fa8e8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 0x7fa8e8003e80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40031a0 tx=0x7fa8f402ef00 comp rx=0 tx=0).stop 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 -- 10.20.193.18:0/3178535174 shutdown_connections 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 --2- 10.20.193.18:0/3178535174 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa8e8005520 0x7fa8e8003e80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 -- 10.20.193.18:0/3178535174 >> 10.20.193.18:0/3178535174 conn(0x7fa9000ce590 msgr2=0x7fa9000ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 -- 10.20.193.18:0/3178535174 shutdown_connections 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 1 -- 10.20.193.18:0/3178535174 wait complete. 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.186 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.172+0000 7fa907914640 10 monclient: monmap: 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 librados: starting msgr at 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 librados: starting objecter 2026-02-24T18:40:49.187 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 Processor -- start 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 -- start start 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 librados: setting wanted keys 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 librados: calling monclient init 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient: init 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa9000c9ef0 0x7fa9000ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): picked mon.noname-a con 0x7fa9000c9ef0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9001f7ab0 0x7fa9000c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): picked mon.noname-b con 0x7fa9001f7ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa9000c8450 0x7fa9001f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.188 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): picked mon.noname-c con 0x7fa9000c8450 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa90015ebd0 con 0x7fa9000c9ef0 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa90015e860 con 0x7fa9001f7ab0 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa90015f060 con 0x7fa9000c8450 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa907914640 10 monclient(hunting): authenticate will time out at 1792.854703s 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.173+0000 7fa905689640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa9000c9ef0 0x7fa9000ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9001f7ab0 0x7fa9000c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa9000c8450 0x7fa9001f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa9000c9ef0 0x7fa9000ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54474/0 (socket says 10.20.193.18:54474) 2026-02-24T18:40:49.189 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 1 -- 10.20.193.18:0/2018673163 learned_addr learned my addr 10.20.193.18:0/2018673163 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): get_auth_request con 0x7fa9000c9ef0 auth_method 0 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): get_auth_request con 0x7fa9001f7ab0 auth_method 0 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): get_auth_request con 0x7fa9000c8450 auth_method 0 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.190 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa904e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.191 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.174+0000 7fa905e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 10 monclient(hunting): handle_auth_done global_id 5267 payload 824 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 -- 10.20.193.18:0/2018673163 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9001f7ab0 msgr2=0x7fa9000c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7fa9001f7ab0 0x7fa9000c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 -- 10.20.193.18:0/2018673163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa9000c9ef0 msgr2=0x7fa9000ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7fa9000c9ef0 0x7fa9000ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 monclient: found mon.noname-c 2026-02-24T18:40:49.195 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.175+0000 7fa905e8a640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000c6b90 con 0x7fa9000c8450 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.176+0000 7fa905e8a640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa9000c8450 0x7fa9001f2a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9000c5430 tx=0x7fa8fc00f5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.176+0000 7fa8edffb640 10 client.?.objecter ms_handle_connect 0x7fa9000c8450 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.176+0000 7fa8edffb640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc019070 con 0x7fa9000c8450 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 10 monclient: dump: 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.196 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa8edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 5 monclient: authenticate success, global_id 5267 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 10 monclient: _renew_subs 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9000c6e70 con 0x7fa9000c8450 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 10 client.5267.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.197 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.177+0000 7fa907914640 10 monclient: _renew_subs 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9000c7430 con 0x7fa9000c8450 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 1 librados: init done 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00fe20 con 0x7fa9000c8450 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc023060 con 0x7fa9000c8450 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8ef7fe640 4 set_mon_vals no callback set 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa8edffb640 10 monclient: dump: 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.198 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 10 client.5267.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 15 client.5267.objecter _session_command_op_assign -1 1 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 20 client.5267.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 10 client.5267.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.178+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9000cd6f0 con 0x7fa9000c8450 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.189+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa8fc026050 con 0x7fa9000c8450 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.189+0000 7fa8edffb640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa8d0078e00 0x7fa8d007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.199 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa905689640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa8d0078e00 0x7fa8d007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa905689640 10 monclient: get_auth_request con 0x7fa8d0078e00 auth_method 0 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa905689640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa8d0078e00 0x7fa8d007b2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00028f0 tx=0x7fa8f00035a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 10 client.5267.objecter ms_handle_connect 0x7fa8d0078e00 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7fa8fc097550 con 0x7fa9000c8450 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 10 client.5267.objecter ms_dispatch2 0x7fa900000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 3 client.5267.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 3 client.5267.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 10 client.5267.objecter checking command 1 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7fa8d00812e0 0x7fa8d0083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter _get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 15 client.5267.objecter _session_command_op_remove -1 1 2026-02-24T18:40:49.200 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter _get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 15 client.5267.objecter _session_command_op_assign 7 1 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 4 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 10 client.5267.objecter _send_command 1 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8d0083dd0 con 0x7fa8d00812e0 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 20 client.5267.objecter dump_active .. 0 homeless 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa8fc05e1b0 con 0x7fa9000c8450 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.190+0000 7fa8edffb640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.191+0000 7fa904e88640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7fa8d00812e0 0x7fa8d0083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.191+0000 7fa904e88640 10 monclient: get_auth_request con 0x7fa8d00812e0 auth_method 0 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.193+0000 7fa904e88640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7fa8d00812e0 0x7fa8d0083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.201 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.193+0000 7fa8edffb640 10 client.5267.objecter ms_handle_connect 0x7fa8d00812e0 2026-02-24T18:40:49.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.203+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== osd.7 v2:10.20.193.112:6810/92089351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa8d0083dd0 con 0x7fa8d00812e0 2026-02-24T18:40:49.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.203+0000 7fa8edffb640 10 client.5267.objecter ms_dispatch2 0x7fa900000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:49.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.203+0000 7fa8edffb640 10 client.5267.objecter _finish_command 1 = system:0 2026-02-24T18:40:49.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.203+0000 7fa8edffb640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 4 2026-02-24T18:40:49.205 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.203+0000 7fa8edffb640 15 client.5267.objecter _session_command_op_remove 7 1 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 10 client.5267.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 15 client.5267.objecter _session_command_op_assign -1 2 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 20 client.5267.objecter _get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 20 client.5267.objecter _get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 15 client.5267.objecter _session_command_op_remove -1 2 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 20 client.5267.objecter get_session s=0x7fa8d0080c10 osd=7 3 2026-02-24T18:40:49.211 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 15 client.5267.objecter _session_command_op_assign 7 2 2026-02-24T18:40:49.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 4 2026-02-24T18:40:49.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 10 client.5267.objecter _send_command 2 2026-02-24T18:40:49.212 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.209+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 --> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa90014ef10 con 0x7fa8d00812e0 2026-02-24T18:40:49.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.223+0000 7fa8edffb640 1 -- 10.20.193.18:0/2018673163 <== osd.7 v2:10.20.193.112:6810/92089351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7fa90014ef10 con 0x7fa8d00812e0 2026-02-24T18:40:49.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.223+0000 7fa8edffb640 10 client.5267.objecter ms_dispatch2 0x7fa900000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:49.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.223+0000 7fa8edffb640 10 client.5267.objecter _finish_command 2 = system:0 2026-02-24T18:40:49.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.223+0000 7fa8edffb640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 4 2026-02-24T18:40:49.225 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.223+0000 7fa8edffb640 15 client.5267.objecter _session_command_op_remove 7 2 2026-02-24T18:40:49.226 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:49.226 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:49.226 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:49.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 10 librados: watch_flush enter 2026-02-24T18:40:49.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 10 librados: watch_flush exit 2026-02-24T18:40:49.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 10 client.5267.objecter close_session for osd.7 2026-02-24T18:40:49.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7fa8d00812e0 msgr2=0x7fa8d0083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.227 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6810/92089351,v1:10.20.193.112:6811/92089351] conn(0x7fa8d00812e0 0x7fa8d0083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 20 client.5267.objecter put_session s=0x7fa8d0080c10 osd=7 1 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 20 client.5267.objecter shutdown clearing up homeless session... 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 10 client.5267.objecter successfully canceled tick 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa8d0078e00 msgr2=0x7fa8d007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7fa8d0078e00 0x7fa8d007b2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00028f0 tx=0x7fa8f00035a0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 10 monclient: shutdown 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa9000c8450 msgr2=0x7fa9001f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 --2- 10.20.193.18:0/2018673163 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7fa9000c8450 0x7fa9001f2a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9000c5430 tx=0x7fa8fc00f5a0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa905e8a640 1 -- 10.20.193.18:0/2018673163 reap_dead start 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 shutdown_connections 2026-02-24T18:40:49.228 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 >> 10.20.193.18:0/2018673163 conn(0x7fa9000ce590 msgr2=0x7fa9000d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 shutdown_connections 2026-02-24T18:40:49.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 -- 10.20.193.18:0/2018673163 wait complete. 2026-02-24T18:40:49.229 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.224+0000 7fa907914640 1 librados: shutdown 2026-02-24T18:40:49.235 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.301+0000 7f7109703640 10 monclient: get_monmap_and_config 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.301+0000 7f7109703640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.301+0000 7f7109703640 10 monclient: monmap: 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:49.302420+0000 2026-02-24T18:40:49.303 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:49.302420+0000 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:49.304 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 Processor -- start 2026-02-24T18:40:49.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 -- start start 2026-02-24T18:40:49.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient: init 2026-02-24T18:40:49.307 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient: _add_conns ranks=[5,3,4,2,0,1] 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f71041527d0 0x7f710406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): picked mon.noname-c con 0x7f71041527d0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): picked mon.noname-a con 0x7f7104152bb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f71040c5fc0 0x7f71040c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): picked mon.noname-b con 0x7f71040c5fc0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f710406f1f0 con 0x7f7104152bb0 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71040c6470 con 0x7f71040c5fc0 2026-02-24T18:40:49.308 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f710406e300 con 0x7f71041527d0 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.302+0000 7f7109703640 10 monclient(hunting): authenticate will time out at 1792.983653s 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f71041527d0 0x7f710406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f71040c5fc0 0x7f71040c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54478/0 (socket says 10.20.193.18:54478) 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 1 -- 10.20.193.18:0/2833123233 learned_addr learned my addr 10.20.193.18:0/2833123233 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): get_auth_request con 0x7f7104152bb0 auth_method 0 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.309 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): get_auth_request con 0x7f71040c5fc0 auth_method 0 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): get_auth_request con 0x7f71041527d0 auth_method 0 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.310 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.303+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 10 monclient(hunting): handle_auth_done global_id 5287 payload 274 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 -- 10.20.193.18:0/2833123233 >> v2:10.20.193.110:3300/0 conn(0x7f71041527d0 msgr2=0x7f710406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 --2- 10.20.193.18:0/2833123233 >> v2:10.20.193.110:3300/0 conn(0x7f71041527d0 0x7f710406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 -- 10.20.193.18:0/2833123233 >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 msgr2=0x7f71040c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 --2- 10.20.193.18:0/2833123233 >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 monclient: found mon.noname-b 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71037fe640 1 -- 10.20.193.18:0/2833123233 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710406e5e0 con 0x7f71040c5fc0 2026-02-24T18:40:49.311 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.304+0000 7f71027fc640 1 --2- 10.20.193.18:0/2833123233 >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-24T18:40:49.312 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.305+0000 7f71037fe640 1 --2- 10.20.193.18:0/2833123233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f71040c5fc0 0x7f71040c8e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7104158940 tx=0x7f70f402f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa9570efc808665 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 1 -- 10.20.193.18:0/2833123233 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f403c070 con 0x7f71040c5fc0 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: dump: 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.318 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 1 -- 10.20.193.18:0/2833123233 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f402fe40 con 0x7f71040c5fc0 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 5 monclient: authenticate success, global_id 5287 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 1 -- 10.20.193.18:0/2833123233 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4037660 con 0x7f71040c5fc0 2026-02-24T18:40:49.319 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7101ffb640 10 monclient: dump: 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.320 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 4 set_mon_vals no callback set 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 10 monclient: shutdown 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 -- 10.20.193.18:0/2833123233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f71040c5fc0 msgr2=0x7f71040c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 --2- 10.20.193.18:0/2833123233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f71040c5fc0 0x7f71040c8e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7104158940 tx=0x7f70f402f1c0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 -- 10.20.193.18:0/2833123233 shutdown_connections 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 --2- 10.20.193.18:0/2833123233 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f71040c5fc0 0x7f71040c8e60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 --2- 10.20.193.18:0/2833123233 >> v2:10.20.193.18:3300/0 conn(0x7f7104152bb0 0x7f71040c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 --2- 10.20.193.18:0/2833123233 >> v2:10.20.193.110:3300/0 conn(0x7f71041527d0 0x7f710406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 -- 10.20.193.18:0/2833123233 >> 10.20.193.18:0/2833123233 conn(0x7f71040cd780 msgr2=0x7f71040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 -- 10.20.193.18:0/2833123233 shutdown_connections 2026-02-24T18:40:49.321 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.307+0000 7f7109703640 1 -- 10.20.193.18:0/2833123233 wait complete. 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 10 monclient: monmap: 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 librados: starting msgr at 2026-02-24T18:40:49.322 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 librados: starting objecter 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 Processor -- start 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 -- start start 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 librados: setting wanted keys 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 1 librados: calling monclient init 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.308+0000 7f7109703640 10 monclient: init 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f71040c5fc0 0x7f71040c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): picked mon.noname-c con 0x7f71040c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 0x7f7104164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): picked mon.noname-a con 0x7f71041527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 0x7f71040c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.323 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): picked mon.noname-b con 0x7f7104152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f710406ee80 con 0x7f71041527d0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f710406eba0 con 0x7f7104152bb0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f710406dfa0 con 0x7f71040c5fc0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7109703640 10 monclient(hunting): authenticate will time out at 1792.990367s 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 0x7f7104164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71037fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 0x7f71040c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7102ffd640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f71040c5fc0 0x7f71040c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.324 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 0x7f7104164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54486/0 (socket says 10.20.193.18:54486) 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 1 -- 10.20.193.18:0/4198586870 learned_addr learned my addr 10.20.193.18:0/4198586870 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 10 monclient(hunting): get_auth_request con 0x7f71041527d0 auth_method 0 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71037fe640 10 monclient(hunting): get_auth_request con 0x7f7104152bb0 auth_method 0 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71037fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f71037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7102ffd640 10 monclient(hunting): get_auth_request con 0x7f71040c5fc0 auth_method 0 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7102ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.325 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.309+0000 7f7102ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f71037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.310+0000 7f7102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 10 monclient(hunting): handle_auth_done global_id 5290 payload 824 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 -- 10.20.193.18:0/4198586870 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f71040c5fc0 msgr2=0x7f71040c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f71040c5fc0 0x7f71040c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.326 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 -- 10.20.193.18:0/4198586870 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 msgr2=0x7f7104164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 0x7f7104164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 monclient: found mon.noname-b 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.311+0000 7f71037fe640 1 -- 10.20.193.18:0/4198586870 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71041f38f0 con 0x7f7104152bb0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.312+0000 7f71037fe640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 0x7f71040c3150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f710406f370 tx=0x7f70f4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.312+0000 7f70e37fe640 10 client.?.objecter ms_handle_connect 0x7f7104152bb0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.312+0000 7f70e37fe640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4047020 con 0x7f7104152bb0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:49.327 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: dump: 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.328 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f4042470 con 0x7f7104152bb0 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 5 monclient: authenticate success, global_id 5290 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _renew_subs 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71041f3b90 con 0x7f7104152bb0 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f403c040 con 0x7f7104152bb0 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f71017fa640 4 set_mon_vals no callback set 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f70e37fe640 10 monclient: dump: 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.329 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 client.5290.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _renew_subs 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041f4150 con 0x7f7104152bb0 2026-02-24T18:40:49.330 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 1 librados: init done 2026-02-24T18:40:49.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.331 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.314+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710406da60 con 0x7f7104152bb0 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.325+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f70f4042860 con 0x7f7104152bb0 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.326+0000 7f70e37fe640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 0x7f70d007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.326+0000 7f7102ffd640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 0x7f70d007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.326+0000 7f7102ffd640 10 monclient: get_auth_request con 0x7f70d0078e00 auth_method 0 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.326+0000 7f7102ffd640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 0x7f70d007b2c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009c40 tx=0x7f70f8009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.332 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.326+0000 7f70e37fe640 10 client.5290.objecter ms_handle_connect 0x7f70d0078e00 2026-02-24T18:40:49.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.327+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f70f4083d60 con 0x7f7104152bb0 2026-02-24T18:40:49.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.327+0000 7f70e37fe640 10 client.5290.objecter ms_dispatch2 0x7f7104000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:49.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.327+0000 7f70e37fe640 3 client.5290.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:49.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.327+0000 7f70e37fe640 3 client.5290.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:49.333 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.327+0000 7f70e37fe640 20 client.5290.objecter dump_active .. 0 homeless 2026-02-24T18:40:49.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.429+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f70f4051070 con 0x7f7104152bb0 2026-02-24T18:40:49.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.429+0000 7f70e37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.430 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.429+0000 7f70e37fe640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:49.511 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.509+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f71040c85e0 con 0x7f70d0078e00 2026-02-24T18:40:49.512 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f70e37fe640 1 -- 10.20.193.18:0/4198586870 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f71040c85e0 con 0x7f70d0078e00 2026-02-24T18:40:49.512 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 7 to deep-scrub 2026-02-24T18:40:49.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 10 librados: watch_flush enter 2026-02-24T18:40:49.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 10 librados: watch_flush exit 2026-02-24T18:40:49.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 20 client.5290.objecter shutdown clearing up homeless session... 2026-02-24T18:40:49.513 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 10 client.5290.objecter successfully canceled tick 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 msgr2=0x7f70d007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 0x7f70d007b2c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009c40 tx=0x7f70f8009290 comp rx=0 tx=0).stop 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 10 monclient: shutdown 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 msgr2=0x7f71040c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 0x7f71040c3150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f710406f370 tx=0x7f70f4004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 shutdown_connections 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f70d0078e00 0x7f70d007b2c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.510+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7104152bb0 0x7f71040c3150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f71041527d0 0x7f7104164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.514 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 --2- 10.20.193.18:0/4198586870 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f71040c5fc0 0x7f71040c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 >> 10.20.193.18:0/4198586870 conn(0x7f71040cd780 msgr2=0x7f71040ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 shutdown_connections 2026-02-24T18:40:49.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 -- 10.20.193.18:0/4198586870 wait complete. 2026-02-24T18:40:49.515 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.511+0000 7f7109703640 1 librados: shutdown 2026-02-24T18:40:49.521 INFO:tasks.ceph:Scrubbing osd.11 2026-02-24T18:40:49.521 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:49.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.589+0000 7f32e496f640 10 monclient: get_monmap_and_config 2026-02-24T18:40:49.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.589+0000 7f32e496f640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.591 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.589+0000 7f32e496f640 10 monclient: monmap: 2026-02-24T18:40:49.591 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.591 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:49.590610+0000 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:49.590610+0000 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:49.592 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 Processor -- start 2026-02-24T18:40:49.595 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 -- start start 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient: init 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient: _add_conns ranks=[0,3,5,1,2,4] 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32dc1503e0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f32dc06fdb0 0x7f32dc0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-a con 0x7f32dc06fdb0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f32dc0c8aa0 0x7f32dc0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-c con 0x7f32dc0c8aa0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc070710 con 0x7f32dc1503e0 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0c7d70 con 0x7f32dc06fdb0 2026-02-24T18:40:49.596 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0c8610 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.590+0000 7f32e496f640 10 monclient(hunting): authenticate will time out at 1793.271772s 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f32dc06fdb0 0x7f32dc0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f32dc0c8aa0 0x7f32dc0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e26e4640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f32dc1503e0 0x7f32dc159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:45982/0 (socket says 10.20.193.18:45982) 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e26e4640 1 -- 10.20.193.18:0/724578078 learned_addr learned my addr 10.20.193.18:0/724578078 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): get_auth_request con 0x7f32dc0c8aa0 auth_method 0 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): get_auth_request con 0x7f32dc06fdb0 auth_method 0 2026-02-24T18:40:49.597 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.591+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_done global_id 5276 payload 274 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 1 -- 10.20.193.18:0/724578078 >> v2:10.20.193.18:3300/0 conn(0x7f32dc06fdb0 msgr2=0x7f32dc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 1 --2- 10.20.193.18:0/724578078 >> v2:10.20.193.18:3300/0 conn(0x7f32dc06fdb0 0x7f32dc0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 1 -- 10.20.193.18:0/724578078 >> v1:10.20.193.18:6789/0 conn(0x7f32dc1503e0 legacy=0x7f32dc159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.598 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 1 monclient: found mon.noname-c 2026-02-24T18:40:49.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.592+0000 7f32e2ee5640 1 -- 10.20.193.18:0/724578078 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0c9420 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.599 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.593+0000 7f32e2ee5640 1 --2- 10.20.193.18:0/724578078 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0c8ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0709b0 tx=0x7f32d002f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13c7214e9dd202f server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 1 -- 10.20.193.18:0/724578078 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d003c070 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: dump: 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.605 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.606 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 1 -- 10.20.193.18:0/724578078 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d0037440 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 5 monclient: authenticate success, global_id 5276 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 1 -- 10.20.193.18:0/724578078 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037810 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 4 set_mon_vals no callback set 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e16e2640 10 monclient: dump: 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.607 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 10 monclient: shutdown 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 -- 10.20.193.18:0/724578078 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 msgr2=0x7f32dc0c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 --2- 10.20.193.18:0/724578078 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0c8ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0709b0 tx=0x7f32d002f250 comp rx=0 tx=0).stop 2026-02-24T18:40:49.608 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 -- 10.20.193.18:0/724578078 shutdown_connections 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 --2- 10.20.193.18:0/724578078 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0c8ee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 --2- 10.20.193.18:0/724578078 >> v2:10.20.193.18:3300/0 conn(0x7f32dc06fdb0 0x7f32dc0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 -- 10.20.193.18:0/724578078 >> 10.20.193.18:0/724578078 conn(0x7f32dc0cd600 msgr2=0x7f32dc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 -- 10.20.193.18:0/724578078 shutdown_connections 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.595+0000 7f32e496f640 1 -- 10.20.193.18:0/724578078 wait complete. 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient: monmap: 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.609 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 librados: starting msgr at 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 librados: starting objecter 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 Processor -- start 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 -- start start 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 librados: setting wanted keys 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 librados: calling monclient init 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient: init 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.610 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f32dc06fdb0 0x7f32dc14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-b con 0x7f32dc06fdb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-c con 0x7f32dc0c8aa0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32dc1503e0 0x7f32dc073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): picked mon.noname-a con 0x7f32dc1503e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0c7fc0 con 0x7f32dc1503e0 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc15a850 con 0x7f32dc06fdb0 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0c82a0 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.611 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.596+0000 7f32e496f640 10 monclient(hunting): authenticate will time out at 1793.277820s 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32dc1503e0 0x7f32dc073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32dc1503e0 0x7f32dc073920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54500/0 (socket says 10.20.193.18:54500) 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 1 -- 10.20.193.18:0/606019936 learned_addr learned my addr 10.20.193.18:0/606019936 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): get_auth_request con 0x7f32dc1503e0 auth_method 0 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): get_auth_request con 0x7f32dc0c8aa0 auth_method 0 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.612 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f32dc06fdb0 0x7f32dc14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e2ee5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): get_auth_request con 0x7f32dc06fdb0 auth_method 0 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.613 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.597+0000 7f32e26e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 10 monclient(hunting): handle_auth_done global_id 5279 payload 824 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 -- 10.20.193.18:0/606019936 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f32dc06fdb0 msgr2=0x7f32dc14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f32dc06fdb0 0x7f32dc14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 -- 10.20.193.18:0/606019936 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32dc1503e0 msgr2=0x7f32dc073920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f32dc1503e0 0x7f32dc073920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 monclient: found mon.noname-c 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.598+0000 7f32e1ee3640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc074360 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.614 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.599+0000 7f32e1ee3640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0725c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e9e0 tx=0x7f32c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.599+0000 7f32c6ffd640 10 client.?.objecter ms_handle_connect 0x7f32dc0c8aa0 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.599+0000 7f32c6ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: dump: 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.615 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c800ca00 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800cdf0 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 5 monclient: authenticate success, global_id 5279 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 monclient: _renew_subs 2026-02-24T18:40:49.616 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e0ee1640 4 set_mon_vals no callback set 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc1555a0 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32c6ffd640 10 monclient: dump: 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.617 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 client.5279.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 monclient: _renew_subs 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc155b60 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 1 librados: init done 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 client.5279.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 15 client.5279.objecter _session_command_op_assign -1 1 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 20 client.5279.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 client.5279.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:40:49.618 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.601+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32dc14c760 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.617+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f32c8026480 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.617+0000 7f32c6ffd640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f32ac078e00 0x7f32ac07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.617+0000 7f32e26e4640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f32ac078e00 0x7f32ac07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.617+0000 7f32e26e4640 10 monclient: get_auth_request con 0x7f32ac078e00 auth_method 0 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.618+0000 7f32e26e4640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f32ac078e00 0x7f32ac07b2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32d4002770 tx=0x7f32d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.620 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.618+0000 7f32c6ffd640 10 client.5279.objecter ms_handle_connect 0x7f32ac078e00 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f32c8097d50 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 10 client.5279.objecter ms_dispatch2 0x7f32dc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 3 client.5279.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 3 client.5279.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 10 client.5279.objecter checking command 1 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7f32ac0812e0 0x7f32ac083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter _get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 15 client.5279.objecter _session_command_op_remove -1 1 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter _get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.622 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 15 client.5279.objecter _session_command_op_assign 11 1 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 4 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 10 client.5279.objecter _send_command 1 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32ac083dd0 con 0x7f32ac0812e0 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 20 client.5279.objecter dump_active .. 0 homeless 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32c805e810 con 0x7f32dc0c8aa0 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32c6ffd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32e2ee5640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7f32ac0812e0 0x7f32ac083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.619+0000 7f32e2ee5640 10 monclient: get_auth_request con 0x7f32ac0812e0 auth_method 0 2026-02-24T18:40:49.623 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.621+0000 7f32c6ffd640 10 client.5279.objecter ms_handle_connect 0x7f32ac0812e0 2026-02-24T18:40:49.624 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.621+0000 7f32e2ee5640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7f32ac0812e0 0x7f32ac083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.631 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.630+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== osd.11 v2:10.20.193.112:6812/2071089960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f32ac083dd0 con 0x7f32ac0812e0 2026-02-24T18:40:49.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.630+0000 7f32c6ffd640 10 client.5279.objecter ms_dispatch2 0x7f32dc000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:49.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.630+0000 7f32c6ffd640 10 client.5279.objecter _finish_command 1 = system:0 2026-02-24T18:40:49.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.630+0000 7f32c6ffd640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 4 2026-02-24T18:40:49.632 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.630+0000 7f32c6ffd640 15 client.5279.objecter _session_command_op_remove 11 1 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 10 client.5279.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 15 client.5279.objecter _session_command_op_assign -1 2 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 20 client.5279.objecter _get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 20 client.5279.objecter _get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 15 client.5279.objecter _session_command_op_remove -1 2 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 20 client.5279.objecter get_session s=0x7f32ac080c10 osd=11 3 2026-02-24T18:40:49.638 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 15 client.5279.objecter _session_command_op_assign 11 2 2026-02-24T18:40:49.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 4 2026-02-24T18:40:49.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 10 client.5279.objecter _send_command 2 2026-02-24T18:40:49.639 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.636+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 --> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f32dc0c5820 con 0x7f32ac0812e0 2026-02-24T18:40:49.651 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32c6ffd640 1 -- 10.20.193.18:0/606019936 <== osd.11 v2:10.20.193.112:6812/2071089960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7f32dc0c5820 con 0x7f32ac0812e0 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32c6ffd640 10 client.5279.objecter ms_dispatch2 0x7f32dc000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32c6ffd640 10 client.5279.objecter _finish_command 2 = system:0 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32c6ffd640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 4 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32c6ffd640 15 client.5279.objecter _session_command_op_remove 11 2 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:40:49.652 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 10 librados: watch_flush enter 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 10 librados: watch_flush exit 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 10 client.5279.objecter close_session for osd.11 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7f32ac0812e0 msgr2=0x7f32ac083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6812/2071089960,v1:10.20.193.112:6813/2071089960] conn(0x7f32ac0812e0 0x7f32ac083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 20 client.5279.objecter put_session s=0x7f32ac080c10 osd=11 1 2026-02-24T18:40:49.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 20 client.5279.objecter shutdown clearing up homeless session... 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 10 client.5279.objecter successfully canceled tick 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f32ac078e00 msgr2=0x7f32ac07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f32ac078e00 0x7f32ac07b2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32d4002770 tx=0x7f32d4009040 comp rx=0 tx=0).stop 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 10 monclient: shutdown 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 msgr2=0x7f32dc0725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 --2- 10.20.193.18:0/606019936 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f32dc0c8aa0 0x7f32dc0725c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32c800e9e0 tx=0x7f32c800eeb0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e2ee5640 1 -- 10.20.193.18:0/606019936 reap_dead start 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 shutdown_connections 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 >> 10.20.193.18:0/606019936 conn(0x7f32dc0cd600 msgr2=0x7f32dc0cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 shutdown_connections 2026-02-24T18:40:49.655 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 -- 10.20.193.18:0/606019936 wait complete. 2026-02-24T18:40:49.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.650+0000 7f32e496f640 1 librados: shutdown 2026-02-24T18:40:49.660 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2026-02-24T18:40:49.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.727+0000 7f23b4443640 10 monclient: get_monmap_and_config 2026-02-24T18:40:49.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.727+0000 7f23b4443640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.729 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.727+0000 7f23b4443640 10 monclient: monmap: 2026-02-24T18:40:49.729 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:49.728734+0000 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:49.728734+0000 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:49.730 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 Processor -- start 2026-02-24T18:40:49.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 -- start start 2026-02-24T18:40:49.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient: init 2026-02-24T18:40:49.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.733 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient: _add_conns ranks=[3,0,4,5,2,1] 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f23ac06ff20 0x7f23ac070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-a con 0x7f23ac06ff20 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23ac0c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f23ac15a5e0 0x7f23ac14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-b con 0x7f23ac15a5e0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac0c7e90 con 0x7f23ac0c99a0 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f23ac159a40 con 0x7f23ac06ff20 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f23ac0c8170 con 0x7f23ac15a5e0 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.734 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.728+0000 7f23b4443640 10 monclient(hunting): authenticate will time out at 1793.409946s 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b21b8640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f23ac06ff20 0x7f23ac070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f23ac15a5e0 0x7f23ac14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b19b7640 1 --1- >> v1:10.20.193.18:6789/0 conn(0x7f23ac0c99a0 0x7f23ac159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.18:6789/0 says I am v1:10.20.193.18:45984/0 (socket says 10.20.193.18:45984) 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b19b7640 1 -- 10.20.193.18:0/75988595 learned_addr learned my addr 10.20.193.18:0/75988595 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b21b8640 10 monclient(hunting): get_auth_request con 0x7f23ac06ff20 auth_method 0 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b21b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b21b8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b21b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): get_auth_request con 0x7f23ac15a5e0 auth_method 0 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.735 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.729+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 10 monclient(hunting): handle_auth_done global_id 5296 payload 274 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 1 -- 10.20.193.18:0/75988595 >> v2:10.20.193.18:3300/0 conn(0x7f23ac06ff20 msgr2=0x7f23ac070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 1 --2- 10.20.193.18:0/75988595 >> v2:10.20.193.18:3300/0 conn(0x7f23ac06ff20 0x7f23ac070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 1 -- 10.20.193.18:0/75988595 >> v1:10.20.193.18:6789/0 conn(0x7f23ac0c99a0 legacy=0x7f23ac159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 1 monclient: found mon.noname-b 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.736 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.737 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.730+0000 7f23b29b9640 1 -- 10.20.193.18:0/75988595 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac0c8450 con 0x7f23ac15a5e0 2026-02-24T18:40:49.737 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.731+0000 7f23b29b9640 1 --2- 10.20.193.18:0/75988595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac14fe10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009d90 tx=0x7f23a802f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9af24f82cc1de5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 1 -- 10.20.193.18:0/75988595 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a803c070 con 0x7f23ac15a5e0 2026-02-24T18:40:49.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:49.743 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: dump: 2026-02-24T18:40:49.743 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 1 -- 10.20.193.18:0/75988595 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a8037440 con 0x7f23ac15a5e0 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 5 monclient: authenticate success, global_id 5296 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 1 -- 10.20.193.18:0/75988595 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8037810 con 0x7f23ac15a5e0 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b11b6640 10 monclient: dump: 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.745 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 4 set_mon_vals no callback set 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 10 monclient: shutdown 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 -- 10.20.193.18:0/75988595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 msgr2=0x7f23ac14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 --2- 10.20.193.18:0/75988595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac14fe10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009d90 tx=0x7f23a802f1e0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 -- 10.20.193.18:0/75988595 shutdown_connections 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 --2- 10.20.193.18:0/75988595 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac14fe10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 --2- 10.20.193.18:0/75988595 >> v2:10.20.193.18:3300/0 conn(0x7f23ac06ff20 0x7f23ac070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 -- 10.20.193.18:0/75988595 >> 10.20.193.18:0/75988595 conn(0x7f23ac0cd780 msgr2=0x7f23ac0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 -- 10.20.193.18:0/75988595 shutdown_connections 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.733+0000 7f23b4443640 1 -- 10.20.193.18:0/75988595 wait complete. 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 10 monclient: build_initial_monmap 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 10 monclient: monmap: 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:49.747 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 librados: starting msgr at 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 librados: starting objecter 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 Processor -- start 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 -- start start 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 librados: setting wanted keys 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 1 librados: calling monclient init 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 10 monclient: init 2026-02-24T18:40:49.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.734+0000 7f23b4443640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 0x7f23ac1f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-a con 0x7f23ac06ff20 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f23ac0c99a0 0x7f23ac0bb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-c con 0x7f23ac0c99a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): picked mon.noname-b con 0x7f23ac15a5e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac150350 con 0x7f23ac06ff20 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac0c8a10 con 0x7f23ac15a5e0 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:49.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1504d0 con 0x7f23ac0c99a0 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b4443640 10 monclient(hunting): authenticate will time out at 1793.416072s 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 0x7f23ac1f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 0x7f23ac1f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54516/0 (socket says 10.20.193.18:54516) 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 1 -- 10.20.193.18:0/1085144539 learned_addr learned my addr 10.20.193.18:0/1085144539 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): get_auth_request con 0x7f23ac06ff20 auth_method 0 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b19b7640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f23ac0c99a0 0x7f23ac0bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): get_auth_request con 0x7f23ac15a5e0 auth_method 0 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b19b7640 10 monclient(hunting): get_auth_request con 0x7f23ac0c99a0 auth_method 0 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b19b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b19b7640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b19b7640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b21b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.735+0000 7f23b29b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.736+0000 7f23b19b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.736+0000 7f23b19b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.736+0000 7f23b19b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 10 monclient(hunting): handle_auth_done global_id 5299 payload 824 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 -- 10.20.193.18:0/1085144539 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f23ac0c99a0 msgr2=0x7f23ac0bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f23ac0c99a0 0x7f23ac0bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 -- 10.20.193.18:0/1085144539 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 msgr2=0x7f23ac1f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 0x7f23ac1f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 10 monclient: _finish_hunting 0 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 monclient: found mon.noname-b 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.737+0000 7f23b29b9640 1 -- 10.20.193.18:0/1085144539 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac0bbe90 con 0x7f23ac15a5e0 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.738+0000 7f23b29b9640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac0bb890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f23ac159bc0 tx=0x7f23a8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.738+0000 7f2392ffd640 10 client.?.objecter ms_handle_connect 0x7f23ac15a5e0 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.738+0000 7f2392ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a803c050 con 0x7f23ac15a5e0 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: dump: 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.753 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 5 monclient: authenticate success, global_id 5299 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _renew_subs 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac0bc170 con 0x7f23ac15a5e0 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 client.5299.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:49.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _renew_subs 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac0bc730 con 0x7f23ac15a5e0 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 1 librados: init done 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a803d040 con 0x7f23ac15a5e0 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8041810 con 0x7f23ac15a5e0 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b09b5640 4 set_mon_vals no callback set 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f2392ffd640 10 monclient: dump: 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:49.755 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:49.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.739+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac150860 con 0x7f23ac15a5e0 2026-02-24T18:40:49.844 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.842+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f23a80419b0 con 0x7f23ac15a5e0 2026-02-24T18:40:49.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.843+0000 7f2392ffd640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 0x7f237c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:49.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.843+0000 7f23b21b8640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 0x7f237c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:49.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.843+0000 7f23b21b8640 10 monclient: get_auth_request con 0x7f237c078e00 auth_method 0 2026-02-24T18:40:49.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.844+0000 7f23b21b8640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 0x7f237c07b2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f239c009d00 tx=0x7f239c009290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:49.845 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.844+0000 7f2392ffd640 10 client.5299.objecter ms_handle_connect 0x7f237c078e00 2026-02-24T18:40:49.846 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.845+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f23a8083fc0 con 0x7f23ac15a5e0 2026-02-24T18:40:49.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.845+0000 7f2392ffd640 10 client.5299.objecter ms_dispatch2 0x7f23ac000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:49.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.845+0000 7f2392ffd640 3 client.5299.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:49.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.845+0000 7f2392ffd640 3 client.5299.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:49.847 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.845+0000 7f2392ffd640 20 client.5299.objecter dump_active .. 0 homeless 2026-02-24T18:40:49.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.857+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f23a8087f80 con 0x7f23ac15a5e0 2026-02-24T18:40:49.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.857+0000 7f2392ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:49.858 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.857+0000 7f2392ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:49.940 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.939+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "11", "target": ["mon-mgr", ""]}) -- 0x7f23ac159570 con 0x7f237c078e00 2026-02-24T18:40:49.941 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.939+0000 7f2392ffd640 1 -- 10.20.193.18:0/1085144539 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 11 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f23ac159570 con 0x7f237c078e00 2026-02-24T18:40:49.941 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 11 to deep-scrub 2026-02-24T18:40:49.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 10 librados: watch_flush enter 2026-02-24T18:40:49.942 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 10 librados: watch_flush exit 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 20 client.5299.objecter shutdown clearing up homeless session... 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 10 client.5299.objecter successfully canceled tick 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 msgr2=0x7f237c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 0x7f237c07b2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f239c009d00 tx=0x7f239c009290 comp rx=0 tx=0).stop 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 10 monclient: shutdown 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 msgr2=0x7f23ac0bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac0bb890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f23ac159bc0 tx=0x7f23a8004b40 comp rx=0 tx=0).stop 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 shutdown_connections 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f237c078e00 0x7f237c07b2c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f23ac15a5e0 0x7f23ac0bb890 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.943 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f23ac0c99a0 0x7f23ac0bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 --2- 10.20.193.18:0/1085144539 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f23ac06ff20 0x7f23ac1f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:49.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 >> 10.20.193.18:0/1085144539 conn(0x7f23ac0cd780 msgr2=0x7f23ac14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:49.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 shutdown_connections 2026-02-24T18:40:49.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 -- 10.20.193.18:0/1085144539 wait complete. 2026-02-24T18:40:49.944 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:49.940+0000 7f23b4443640 1 librados: shutdown 2026-02-24T18:40:49.950 INFO:tasks.ceph:Scrubbing osd.15 2026-02-24T18:40:49.950 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:40:50.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 10 monclient: get_monmap_and_config 2026-02-24T18:40:50.024 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 10 monclient: monmap: 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:50.021887+0000 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:50.021887+0000 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:50.025 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 1 Processor -- start 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 1 -- start start 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.021+0000 7f56c44a6640 10 monclient: init 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient: _add_conns ranks=[5,1,4,3,0,2] 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f56bc1503e0 0x7f56bc159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-c con 0x7f56bc1503e0 addr v2:10.20.193.110:3300/0 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f56bc15a070 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:50.026 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f56bc0704c0 0x7f56bc0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-b con 0x7f56bc0704c0 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56bc15a570 con 0x7f56bc15a070 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0c7d70 con 0x7f56bc0704c0 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 1 -- --> v2:10.20.193.110:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0c8610 con 0x7f56bc1503e0 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c44a6640 10 monclient(hunting): authenticate will time out at 1793.703079s 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 1 --2- >> v2:10.20.193.110:3300/0 conn(0x7f56bc1503e0 0x7f56bc159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c1a1a640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f56bc15a070 0x7f56bc06fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:43120/0 (socket says 10.20.193.18:43120) 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c1a1a640 1 -- 10.20.193.18:0/4192056219 learned_addr learned my addr 10.20.193.18:0/4192056219 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:50.027 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 10 monclient(hunting): get_auth_request con 0x7f56bc1503e0 auth_method 0 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.022+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.028 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 10 monclient(hunting): handle_auth_done global_id 5285 payload 274 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 1 -- 10.20.193.18:0/4192056219 >> v2:10.20.193.32:3300/0 conn(0x7f56bc0704c0 msgr2=0x7f56bc0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 1 --2- 10.20.193.18:0/4192056219 >> v2:10.20.193.32:3300/0 conn(0x7f56bc0704c0 0x7f56bc0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 1 -- 10.20.193.18:0/4192056219 >> v1:10.20.193.32:6789/0 conn(0x7f56bc15a070 legacy=0x7f56bc06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 1 monclient: found mon.noname-c 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.023+0000 7f56c221b640 1 -- 10.20.193.18:0/4192056219 --> v2:10.20.193.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc15a850 con 0x7f56bc1503e0 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.025+0000 7f56c221b640 1 --2- 10.20.193.18:0/4192056219 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc1503e0 0x7f56bc159b30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ce490 tx=0x7f56ac02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e07b815a185ab9 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 1 -- 10.20.193.18:0/4192056219 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac03c070 con 0x7f56bc1503e0 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: dump: 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.035 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 1 -- 10.20.193.18:0/4192056219 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac02fb80 con 0x7f56bc1503e0 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 5 monclient: authenticate success, global_id 5285 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 1 -- 10.20.193.18:0/4192056219 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac037440 con 0x7f56bc1503e0 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c1219640 10 monclient: dump: 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.036 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 4 set_mon_vals no callback set 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 10 monclient: shutdown 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 -- 10.20.193.18:0/4192056219 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc1503e0 msgr2=0x7f56bc159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 --2- 10.20.193.18:0/4192056219 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc1503e0 0x7f56bc159b30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ce490 tx=0x7f56ac02eeb0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 -- 10.20.193.18:0/4192056219 shutdown_connections 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 --2- 10.20.193.18:0/4192056219 >> v2:10.20.193.32:3300/0 conn(0x7f56bc0704c0 0x7f56bc0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 --2- 10.20.193.18:0/4192056219 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc1503e0 0x7f56bc159b30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 -- 10.20.193.18:0/4192056219 >> 10.20.193.18:0/4192056219 conn(0x7f56bc0cd600 msgr2=0x7f56bc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 -- 10.20.193.18:0/4192056219 shutdown_connections 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.027+0000 7f56c44a6640 1 -- 10.20.193.18:0/4192056219 wait complete. 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient: monmap: 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:50.038 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 librados: starting msgr at 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 librados: starting objecter 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 Processor -- start 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 -- start start 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 librados: setting wanted keys 2026-02-24T18:40:50.039 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 librados: calling monclient init 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient: init 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc0704c0 0x7f56bc1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-c con 0x7f56bc0704c0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 0x7f56bc1f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-b con 0x7f56bc1503e0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56bc15a070 0x7f56bc1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): picked mon.noname-a con 0x7f56bc15a070 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1507c0 con 0x7f56bc15a070 2026-02-24T18:40:50.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0c82a0 con 0x7f56bc1503e0 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc150940 con 0x7f56bc0704c0 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.028+0000 7f56c44a6640 10 monclient(hunting): authenticate will time out at 1793.709959s 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc0704c0 0x7f56bc1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 0x7f56bc1f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56bc15a070 0x7f56bc1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc0704c0 0x7f56bc1590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:53356/0 (socket says 10.20.193.18:53356) 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 0x7f56bc1f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:51938/0 (socket says 10.20.193.18:51938) 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 1 -- 10.20.193.18:0/2896649751 learned_addr learned my addr 10.20.193.18:0/2896649751 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:50.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): get_auth_request con 0x7f56bc0704c0 auth_method 0 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): get_auth_request con 0x7f56bc1503e0 auth_method 0 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): get_auth_request con 0x7f56bc15a070 auth_method 0 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c221b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c1a1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.029+0000 7f56c2a1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 10 monclient(hunting): handle_auth_done global_id 5305 payload 824 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 -- 10.20.193.18:0/2896649751 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc0704c0 msgr2=0x7f56bc1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.043 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56bc0704c0 0x7f56bc1590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 -- 10.20.193.18:0/2896649751 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56bc15a070 msgr2=0x7f56bc1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56bc15a070 0x7f56bc1f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 monclient: found mon.noname-b 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.031+0000 7f56c1a1a640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1f7110 con 0x7f56bc1503e0 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.032+0000 7f56c1a1a640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 0x7f56bc1f41d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c410 tx=0x7f56b000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.032+0000 7f56a2ffd640 10 client.?.objecter ms_handle_connect 0x7f56bc1503e0 2026-02-24T18:40:50.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.032+0000 7f56a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0017070 con 0x7f56bc1503e0 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: dump: 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.046 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b0005de0 con 0x7f56bc1503e0 2026-02-24T18:40:50.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 5 monclient: authenticate success, global_id 5305 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00072d0 con 0x7f56bc1503e0 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56a2ffd640 10 monclient: dump: 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.048 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c0a18640 4 set_mon_vals no callback set 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 monclient: _renew_subs 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc1f73f0 con 0x7f56bc1503e0 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 client.5305.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 monclient: _renew_subs 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1f79b0 con 0x7f56bc1503e0 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 1 librados: init done 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 client.5305.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 15 client.5305.objecter _session_command_op_assign -1 1 2026-02-24T18:40:50.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 20 client.5305.objecter _get_session osd=-1 returning homeless 2026-02-24T18:40:50.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 client.5305.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:50.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.034+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56bc14dc00 con 0x7f56bc1503e0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.047+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f56b0007470 con 0x7f56bc1503e0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.048+0000 7f56a2ffd640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f568c078e00 0x7f568c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.048+0000 7f56c221b640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f568c078e00 0x7f568c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.048+0000 7f56c221b640 10 monclient: get_auth_request con 0x7f568c078e00 auth_method 0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56c221b640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f568c078e00 0x7f568c07b2c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56ac02eef0 tx=0x7f56ac033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 10 client.5305.objecter ms_handle_connect 0x7f568c078e00 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f56b0007790 con 0x7f56bc1503e0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 10 client.5305.objecter ms_dispatch2 0x7f56bc000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 3 client.5305.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:50.052 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 3 client.5305.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 10 client.5305.objecter checking command 1 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f568c081340 0x7f568c083760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter _get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 15 client.5305.objecter _session_command_op_remove -1 1 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter _get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 15 client.5305.objecter _session_command_op_assign 15 1 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 4 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 10 client.5305.objecter _send_command 1 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56b0009e50 con 0x7f568c081340 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 20 client.5305.objecter dump_active .. 0 homeless 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56b005daa0 con 0x7f56bc1503e0 2026-02-24T18:40:50.053 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.049+0000 7f56a2ffd640 10 monclient: handle_get_version_reply finishing 1 version 29 2026-02-24T18:40:50.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.050+0000 7f56c2a1c640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f568c081340 0x7f568c083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.050+0000 7f56c2a1c640 10 monclient: get_auth_request con 0x7f568c081340 auth_method 0 2026-02-24T18:40:50.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.053+0000 7f56a2ffd640 10 client.5305.objecter ms_handle_connect 0x7f568c081340 2026-02-24T18:40:50.054 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.053+0000 7f56c2a1c640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f568c081340 0x7f568c083760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.062+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== osd.15 v2:10.20.193.112:6802/4220209204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f56b0009e50 con 0x7f568c081340 2026-02-24T18:40:50.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.062+0000 7f56a2ffd640 10 client.5305.objecter ms_dispatch2 0x7f56bc000b90 command_reply(tid 1: 0 ) 2026-02-24T18:40:50.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.062+0000 7f56a2ffd640 10 client.5305.objecter _finish_command 1 = system:0 2026-02-24T18:40:50.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.062+0000 7f56a2ffd640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 4 2026-02-24T18:40:50.064 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.062+0000 7f56a2ffd640 15 client.5305.objecter _session_command_op_remove 15 1 2026-02-24T18:40:50.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 10 client.5305.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-24T18:40:50.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 15 client.5305.objecter _session_command_op_assign -1 2 2026-02-24T18:40:50.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 20 client.5305.objecter _get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.070 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 20 client.5305.objecter _get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 15 client.5305.objecter _session_command_op_remove -1 2 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 20 client.5305.objecter get_session s=0x7f568c080c70 osd=15 3 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 15 client.5305.objecter _session_command_op_assign 15 2 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 4 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 10 client.5305.objecter _send_command 2 2026-02-24T18:40:50.071 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.068+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 --> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f56bc159b30 con 0x7f568c081340 2026-02-24T18:40:50.076 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56a2ffd640 1 -- 10.20.193.18:0/2896649751 <== osd.15 v2:10.20.193.112:6802/4220209204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f56bc159b30 con 0x7f568c081340 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56a2ffd640 10 client.5305.objecter ms_dispatch2 0x7f56bc000b90 command_reply(tid 2: 0 ) 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56a2ffd640 10 client.5305.objecter _finish_command 2 = system:0 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56a2ffd640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 4 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56a2ffd640 15 client.5305.objecter _session_command_op_remove 15 2 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stdout:{ 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T18:40:50.077 INFO:teuthology.orchestra.run.trial018.stdout:} 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 10 librados: watch_flush enter 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 10 librados: watch_flush exit 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 10 client.5305.objecter close_session for osd.15 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f568c081340 msgr2=0x7f568c083760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6802/4220209204,v1:10.20.193.112:6803/4220209204] conn(0x7f568c081340 0x7f568c083760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 20 client.5305.objecter put_session s=0x7f568c080c70 osd=15 1 2026-02-24T18:40:50.079 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 20 client.5305.objecter shutdown clearing up homeless session... 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 10 client.5305.objecter successfully canceled tick 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f568c078e00 msgr2=0x7f568c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f568c078e00 0x7f568c07b2c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56ac02eef0 tx=0x7f56ac033000 comp rx=0 tx=0).stop 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 10 monclient: shutdown 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 msgr2=0x7f56bc1f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 --2- 10.20.193.18:0/2896649751 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56bc1503e0 0x7f56bc1f41d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c410 tx=0x7f56b000c8e0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c2a1c640 1 -- 10.20.193.18:0/2896649751 reap_dead start 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 shutdown_connections 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 >> 10.20.193.18:0/2896649751 conn(0x7f56bc0cd600 msgr2=0x7f56bc0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 shutdown_connections 2026-02-24T18:40:50.080 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 -- 10.20.193.18:0/2896649751 wait complete. 2026-02-24T18:40:50.081 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.075+0000 7f56c44a6640 1 librados: shutdown 2026-02-24T18:40:50.086 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2026-02-24T18:40:50.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.153+0000 7f7ade8f5640 10 monclient: get_monmap_and_config 2026-02-24T18:40:50.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.153+0000 7f7ade8f5640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.155 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.153+0000 7f7ade8f5640 10 monclient: monmap: 2026-02-24T18:40:50.155 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.155 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:50.154654+0000 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:50.154654+0000 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:50.156 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 Processor -- start 2026-02-24T18:40:50.159 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 -- start start 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient: init 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient: _add_conns ranks=[3,0,4,1,2,5] 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7ad81527f0 0x7f7ad8073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-a con 0x7f7ad81527f0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7ad815d2c0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7ad815d730 0x7f7ad80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-b con 0x7f7ad815d730 addr v2:10.20.193.32:3300/0 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad8073c20 con 0x7f7ad815d2c0 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad8074b10 con 0x7f7ad81527f0 2026-02-24T18:40:50.160 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad8073f00 con 0x7f7ad815d730 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.154+0000 7f7ade8f5640 10 monclient(hunting): authenticate will time out at 1793.835992s 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7ad7fff640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f7ad81527f0 0x7f7ad8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7ad815d730 0x7f7ad80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f7ad815d730 0x7f7ad80ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.32:3300/0 says I am v2:10.20.193.18:51942/0 (socket says 10.20.193.18:51942) 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 1 -- 10.20.193.18:0/1903013070 learned_addr learned my addr 10.20.193.18:0/1903013070 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 10 monclient(hunting): get_auth_request con 0x7f7ad815d730 auth_method 0 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7adce6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7ad7fff640 10 monclient(hunting): get_auth_request con 0x7f7ad81527f0 auth_method 0 2026-02-24T18:40:50.161 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7ad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7ad7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.155+0000 7f7ad7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.156+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 10 monclient(hunting): handle_auth_done global_id 5308 payload 274 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 1 -- 10.20.193.18:0/1903013070 >> v2:10.20.193.18:3300/0 conn(0x7f7ad81527f0 msgr2=0x7f7ad8073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 1 --2- 10.20.193.18:0/1903013070 >> v2:10.20.193.18:3300/0 conn(0x7f7ad81527f0 0x7f7ad8073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 1 -- 10.20.193.18:0/1903013070 >> v1:10.20.193.18:6789/0 conn(0x7f7ad815d2c0 legacy=0x7f7ad815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.162 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 1 monclient: found mon.noname-b 2026-02-24T18:40:50.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.157+0000 7f7adce6b640 1 -- 10.20.193.18:0/1903013070 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad80741e0 con 0x7f7ad815d730 2026-02-24T18:40:50.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.158+0000 7f7adce6b640 1 --2- 10.20.193.18:0/1903013070 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d730 0x7f7ad80ccf70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ad806cdf0 tx=0x7f7acc02f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1fca1a999cadc5 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 1 -- 10.20.193.18:0/1903013070 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc02fd70 con 0x7f7ad815d730 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 10 monclient: dump: 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.170 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ad6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.171 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ade8f5640 5 monclient: authenticate success, global_id 5308 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.159+0000 7f7ade8f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 1 -- 10.20.193.18:0/1903013070 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc0373f0 con 0x7f7ad815d730 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 1 -- 10.20.193.18:0/1903013070 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0377c0 con 0x7f7ad815d730 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ad6ffd640 10 monclient: dump: 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.172 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 4 set_mon_vals no callback set 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 10 monclient: shutdown 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 -- 10.20.193.18:0/1903013070 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d730 msgr2=0x7f7ad80ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/1903013070 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d730 0x7f7ad80ccf70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ad806cdf0 tx=0x7f7acc02f270 comp rx=0 tx=0).stop 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 -- 10.20.193.18:0/1903013070 shutdown_connections 2026-02-24T18:40:50.173 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/1903013070 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d730 0x7f7ad80ccf70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/1903013070 >> v2:10.20.193.18:3300/0 conn(0x7f7ad81527f0 0x7f7ad8073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 -- 10.20.193.18:0/1903013070 >> 10.20.193.18:0/1903013070 conn(0x7f7ad80ce730 msgr2=0x7f7ad80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 -- 10.20.193.18:0/1903013070 shutdown_connections 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 1 -- 10.20.193.18:0/1903013070 wait complete. 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.160+0000 7f7ade8f5640 10 monclient: monmap: 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.174 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 librados: starting msgr at 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 librados: starting objecter 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 Processor -- start 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 -- start start 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 librados: setting wanted keys 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 librados: calling monclient init 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient: init 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.175 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-a con 0x7f7ad81527f0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d2c0 0x7f7ad8150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-b con 0x7f7ad815d2c0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7ad815d730 0x7f7ad814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): picked mon.noname-c con 0x7f7ad815d730 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8074fa0 con 0x7f7ad81527f0 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad80747a0 con 0x7f7ad815d2c0 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8075120 con 0x7f7ad815d730 2026-02-24T18:40:50.176 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ade8f5640 10 monclient(hunting): authenticate will time out at 1793.842553s 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ad7fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7adce6b640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7ad815d730 0x7f7ad814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.161+0000 7f7ad77fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d2c0 0x7f7ad8150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54552/0 (socket says 10.20.193.18:54552) 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 1 -- 10.20.193.18:0/832147073 learned_addr learned my addr 10.20.193.18:0/832147073 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): get_auth_request con 0x7f7ad81527f0 auth_method 0 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): get_auth_request con 0x7f7ad815d2c0 auth_method 0 2026-02-24T18:40:50.177 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): get_auth_request con 0x7f7ad815d730 auth_method 0 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7ad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.178 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.162+0000 7f7adce6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 10 monclient(hunting): handle_auth_done global_id 5394 payload 824 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 -- 10.20.193.18:0/832147073 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7ad815d730 msgr2=0x7f7ad814b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7ad815d730 0x7f7ad814b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 -- 10.20.193.18:0/832147073 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d2c0 msgr2=0x7f7ad8150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d2c0 0x7f7ad8150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 monclient: found mon.noname-a 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.163+0000 7f7ad7fff640 1 -- 10.20.193.18:0/832147073 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad814a030 con 0x7f7ad81527f0 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.164+0000 7f7ad7fff640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400ec10 tx=0x7f7ac400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.179 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.164+0000 7f7ad4ff9640 10 client.?.objecter ms_handle_connect 0x7f7ad81527f0 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.164+0000 7f7ad4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4017070 con 0x7f7ad81527f0 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: dump: 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.180 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 5 monclient: authenticate success, global_id 5394 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac400cc40 con 0x7f7ad81527f0 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _renew_subs 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad814a310 con 0x7f7ad81527f0 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4013470 con 0x7f7ad81527f0 2026-02-24T18:40:50.181 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad67fc640 4 set_mon_vals no callback set 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ad4ff9640 10 monclient: dump: 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.182 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 client.5394.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _renew_subs 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad814a8d0 con 0x7f7ad81527f0 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 1 librados: init done 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:40:50.183 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.166+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad8152bd0 con 0x7f7ad81527f0 2026-02-24T18:40:50.184 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.178+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7ac4013630 con 0x7f7ad81527f0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.178+0000 7f7ad4ff9640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 0x7f7aa807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.178+0000 7f7ad77fe640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 0x7f7aa807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.178+0000 7f7ad77fe640 10 monclient: get_auth_request con 0x7f7aa8078e50 auth_method 0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.179+0000 7f7ad77fe640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 0x7f7aa807b310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ad81606f0 tx=0x7f7ac8007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.179+0000 7f7ad4ff9640 10 client.5394.objecter ms_handle_connect 0x7f7aa8078e50 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.180+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f7ac405e560 con 0x7f7ad81527f0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.180+0000 7f7ad4ff9640 10 client.5394.objecter ms_dispatch2 0x7f7ad8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.180+0000 7f7ad4ff9640 3 client.5394.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.180+0000 7f7ad4ff9640 3 client.5394.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:50.185 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.180+0000 7f7ad4ff9640 20 client.5394.objecter dump_active .. 0 homeless 2026-02-24T18:40:50.282 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.281+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mon.0 v2:10.20.193.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7ac4062520 con 0x7f7ad81527f0 2026-02-24T18:40:50.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.281+0000 7f7ad4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.283 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.281+0000 7f7ad4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:50.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.367+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "15", "target": ["mon-mgr", ""]}) -- 0x7f7ad806c030 con 0x7f7aa8078e50 2026-02-24T18:40:50.369 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.368+0000 7f7ad4ff9640 1 -- 10.20.193.18:0/832147073 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 15 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f7ad806c030 con 0x7f7aa8078e50 2026-02-24T18:40:50.370 INFO:teuthology.orchestra.run.trial018.stderr:instructed osd(s) 15 to deep-scrub 2026-02-24T18:40:50.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 10 librados: watch_flush enter 2026-02-24T18:40:50.371 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 10 librados: watch_flush exit 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 20 client.5394.objecter shutdown clearing up homeless session... 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 10 client.5394.objecter successfully canceled tick 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 msgr2=0x7f7aa807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 0x7f7aa807b310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ad81606f0 tx=0x7f7ac8007f30 comp rx=0 tx=0).stop 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 10 monclient: shutdown 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 msgr2=0x7f7ad81f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400ec10 tx=0x7f7ac400c520 comp rx=0 tx=0).stop 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 shutdown_connections 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f7aa8078e50 0x7f7aa807b310 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.372 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f7ad815d2c0 0x7f7ad8150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f7ad815d730 0x7f7ad814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 --2- 10.20.193.18:0/832147073 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f7ad81527f0 0x7f7ad81f63f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 >> 10.20.193.18:0/832147073 conn(0x7f7ad80ce730 msgr2=0x7f7ad80d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 shutdown_connections 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 -- 10.20.193.18:0/832147073 wait complete. 2026-02-24T18:40:50.373 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.369+0000 7f7ade8f5640 1 librados: shutdown 2026-02-24T18:40:50.380 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:40:50.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.446+0000 7f83aea6d640 10 monclient: get_monmap_and_config 2026-02-24T18:40:50.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.446+0000 7f83aea6d640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.448 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.446+0000 7f83aea6d640 10 monclient: monmap: 2026-02-24T18:40:50.448 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:40:50.447729+0000 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:40:50.447729+0000 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:40:50.449 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 Processor -- start 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 -- start start 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient: init 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient: _add_conns ranks=[1,0,3,5,4,2] 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f83a806ff20 addr v1:10.20.193.32:6789/0 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f83a80c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f83a815a5e0 0x7f83a8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.452 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-a con 0x7f83a815a5e0 addr v2:10.20.193.18:3300/0 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a80c7e90 con 0x7f83a80c99a0 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a80c8170 con 0x7f83a806ff20 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 1 -- --> v2:10.20.193.18:3300/0 -- mon_getmap magic: 0 -- 0x7f83a8159a40 con 0x7f83a815a5e0 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.447+0000 7f83aea6d640 10 monclient(hunting): authenticate will time out at 1794.128954s 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83acfe3640 1 --2- >> v2:10.20.193.18:3300/0 conn(0x7f83a815a5e0 0x7f83a8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83a7fff640 1 --1- >> v1:10.20.193.32:6789/0 conn(0x7f83a806ff20 0x7f83a8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.32:6789/0 says I am v1:10.20.193.18:43130/0 (socket says 10.20.193.18:43130) 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83a7fff640 1 -- 10.20.193.18:0/2291930998 learned_addr learned my addr 10.20.193.18:0/2291930998 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83acfe3640 10 monclient(hunting): get_auth_request con 0x7f83a815a5e0 auth_method 0 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83acfe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.453 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83acfe3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.448+0000 7f83acfe3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 1 -- 10.20.193.18:0/2291930998 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142140463 0 0) 0x7f83a80c8170 con 0x7f83a806ff20 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 10 monclient(hunting): _negotiate 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 10 monclient(hunting): my global_id is 5314 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.449+0000 7f83a6ffd640 1 -- 10.20.193.18:0/2291930998 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c003620 con 0x7f83a806ff20 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 10 monclient(hunting): handle_auth_done global_id 5397 payload 274 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 1 -- 10.20.193.18:0/2291930998 >> v1:10.20.193.32:6789/0 conn(0x7f83a806ff20 legacy=0x7f83a8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.454 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 1 -- 10.20.193.18:0/2291930998 >> v1:10.20.193.18:6789/0 conn(0x7f83a80c99a0 legacy=0x7f83a814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 1 monclient: found mon.noname-a 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.18:3300/0 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.450+0000 7f83acfe3640 1 -- 10.20.193.18:0/2291930998 --> v2:10.20.193.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a80c8450 con 0x7f83a815a5e0 2026-02-24T18:40:50.455 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.452+0000 7f83acfe3640 1 --2- 10.20.193.18:0/2291930998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a8070300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f839c009dc0 tx=0x7f839c02f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e4ce34f3ab831d server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.457 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.454+0000 7f83a6ffd640 1 -- 10.20.193.18:0/2291930998 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c03c070 con 0x7f83a815a5e0 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.454+0000 7f83a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.454+0000 7f83a6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.454+0000 7f83a6ffd640 10 monclient: dump: 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.458 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 1 -- 10.20.193.18:0/2291930998 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c037440 con 0x7f83a815a5e0 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 5 monclient: authenticate success, global_id 5397 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 1 -- 10.20.193.18:0/2291930998 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c037810 con 0x7f83a815a5e0 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.459 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83a6ffd640 10 monclient: dump: 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 10 monclient: get_monmap_and_config success 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 4 set_mon_vals no callback set 2026-02-24T18:40:50.460 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 10 monclient: shutdown 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 -- 10.20.193.18:0/2291930998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 msgr2=0x7f83a8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 --2- 10.20.193.18:0/2291930998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a8070300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f839c009dc0 tx=0x7f839c02f280 comp rx=0 tx=0).stop 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 -- 10.20.193.18:0/2291930998 shutdown_connections 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 --2- 10.20.193.18:0/2291930998 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a8070300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 -- 10.20.193.18:0/2291930998 >> 10.20.193.18:0/2291930998 conn(0x7f83a80cd780 msgr2=0x7f83a80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 -- 10.20.193.18:0/2291930998 shutdown_connections 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.455+0000 7f83aea6d640 1 -- 10.20.193.18:0/2291930998 wait complete. 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient: build_initial_monmap 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient: monmap: 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:40:50.461 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 librados: starting msgr at 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 librados: starting objecter 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 Processor -- start 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 -- start start 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 librados: setting wanted keys 2026-02-24T18:40:50.462 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 librados: calling monclient init 2026-02-24T18:40:50.463 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient: init 2026-02-24T18:40:50.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient: _reopen_session rank -1 2026-02-24T18:40:50.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:40:50.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f83a806ff20 0x7f83a81e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.467 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-c con 0x7f83a806ff20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 0x7f83a8147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-b con 0x7f83a80c99a0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a81e9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): picked mon.noname-a con 0x7f83a815a5e0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8150520 con 0x7f83a815a5e0 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80c8a10 con 0x7f83a80c99a0 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): start opening mon connection 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81506a0 con 0x7f83a806ff20 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): _renew_subs 2026-02-24T18:40:50.468 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.456+0000 7f83aea6d640 10 monclient(hunting): authenticate will time out at 1794.137880s 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a81e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 0x7f83a8147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f83a806ff20 0x7f83a81e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a81e9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:54570/0 (socket says 10.20.193.18:54570) 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 1 -- 10.20.193.18:0/1366918094 learned_addr learned my addr 10.20.193.18:0/1366918094 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): get_auth_request con 0x7f83a815a5e0 auth_method 0 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): get_auth_request con 0x7f83a80c99a0 auth_method 0 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.469 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): get_auth_request con 0x7f83a806ff20 auth_method 0 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83acfe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:40:50.470 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.457+0000 7f83a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 10 monclient(hunting): handle_auth_done global_id 5317 payload 824 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 -- 10.20.193.18:0/1366918094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f83a806ff20 msgr2=0x7f83a81e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f83a806ff20 0x7f83a81e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 -- 10.20.193.18:0/1366918094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 msgr2=0x7f83a81e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a81e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 10 monclient: _finish_hunting 0 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 monclient: found mon.noname-b 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.459+0000 7f83a77fe640 1 -- 10.20.193.18:0/1366918094 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81f6bc0 con 0x7f83a80c99a0 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.460+0000 7f83a77fe640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 0x7f83a8147c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f839800c4e0 tx=0x7f839800c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.460+0000 7f83a4ff9640 10 client.?.objecter ms_handle_connect 0x7f83a80c99a0 2026-02-24T18:40:50.471 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.460+0000 7f83a4ff9640 10 client.?.objecter resend_mon_ops 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980052d0 con 0x7f83a80c99a0 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: dump: 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.472 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: _finish_auth 0 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: _check_auth_tickets 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8398005470 con 0x7f83a80c99a0 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398003910 con 0x7f83a80c99a0 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a4ff9640 10 monclient: dump: 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:40:50.473 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83a67fc640 4 set_mon_vals no callback set 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 5 monclient: authenticate success, global_id 5317 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 10 monclient: _renew_subs 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.474 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81e84e0 con 0x7f83a80c99a0 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 10 client.5317.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 10 monclient: _renew_subs 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81e8aa0 con 0x7f83a80c99a0 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.461+0000 7f83aea6d640 1 librados: init done 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.462+0000 7f83aea6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.462+0000 7f83aea6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.462+0000 7f83aea6d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:40:50.475 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.462+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a8070300 con 0x7f83a80c99a0 2026-02-24T18:40:50.476 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.472+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8398003ab0 con 0x7f83a80c99a0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.473+0000 7f83a4ff9640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 0x7f837807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.473+0000 7f83a7fff640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 0x7f837807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.473+0000 7f83a7fff640 10 monclient: get_auth_request con 0x7f8378078e30 auth_method 0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a7fff640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 0x7f837807b2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8394007b60 tx=0x7f8394007ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 10 client.5317.objecter ms_handle_connect 0x7f8378078e30 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f83980977b0 con 0x7f83a80c99a0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 10 client.5317.objecter ms_dispatch2 0x7f83a8000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 3 client.5317.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 3 client.5317.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:40:50.477 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.474+0000 7f83a4ff9640 20 client.5317.objecter dump_active .. 0 homeless 2026-02-24T18:40:50.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.571+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f83a81f6bc0 con 0x7f83a80c99a0 2026-02-24T18:40:50.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.571+0000 7f83a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:40:50.572 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.571+0000 7f83a4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:40:50.653 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.651+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83a81573f0 con 0x7f8378078e30 2026-02-24T18:40:50.654 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83a4ff9640 1 -- 10.20.193.18:0/1366918094 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44705 (secure 0 0 0) 0x7f83a81573f0 con 0x7f8378078e30 2026-02-24T18:40:50.655 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:40:50.656 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:40:50.656 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 10 librados: watch_flush enter 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 10 librados: watch_flush exit 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 20 client.5317.objecter shutdown clearing up homeless session... 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 10 client.5317.objecter successfully canceled tick 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 msgr2=0x7f837807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 0x7f837807b2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8394007b60 tx=0x7f8394007ab0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 10 monclient: shutdown 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 msgr2=0x7f83a8147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 0x7f83a8147c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f839800c4e0 tx=0x7f839800c9b0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 shutdown_connections 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f8378078e30 0x7f837807b2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.657 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f83a80c99a0 0x7f83a8147c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f83a815a5e0 0x7f83a81e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 --2- 10.20.193.18:0/1366918094 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f83a806ff20 0x7f83a81e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 >> 10.20.193.18:0/1366918094 conn(0x7f83a80cd780 msgr2=0x7f83a80c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 shutdown_connections 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 -- 10.20.193.18:0/1366918094 wait complete. 2026-02-24T18:40:50.658 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:40:50.653+0000 7f83aea6d640 1 librados: shutdown 2026-02-24T18:40:50.664 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-24T18:40:49.858180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437128,"kb_used_data":5720,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512312,"statfs":{"total":1546188226560,"available":1545740607488,"internally_reserved":0,"allocated":5857280,"data_stored":2838502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":103,"apply_latency_ms":103,"commit_latency_ns":103000000,"apply_latency_ns":103000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001901"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:43.017956+0000","last_change":"2026-02-24T18:35:32.818354+0000","last_active":"2026-02-24T18:37:43.017956+0000","last_peered":"2026-02-24T18:37:43.017956+0000","last_clean":"2026-02-24T18:37:43.017956+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:37:43.017956+0000","last_undegraded":"2026-02-24T18:37:43.017956+0000","last_fullsized":"2026-02-24T18:37:43.017956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:44:55.242632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:41.029076+0000","last_change":"2026-02-24T18:35:32.673499+0000","last_active":"2026-02-24T18:37:41.029076+0000","last_peered":"2026-02-24T18:37:41.029076+0000","last_clean":"2026-02-24T18:37:41.029076+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:37:41.029076+0000","last_undegraded":"2026-02-24T18:37:41.029076+0000","last_fullsized":"2026-02-24T18:37:41.029076+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:23.258614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:47.353233+0000","last_change":"2026-02-24T18:40:47.357852+0000","last_active":"2026-02-24T18:40:47.353233+0000","last_peered":"2026-02-24T18:40:47.353233+0000","last_clean":"2026-02-24T18:40:47.353233+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:40:47.353233+0000","last_undegraded":"2026-02-24T18:40:47.353233+0000","last_fullsized":"2026-02-24T18:40:47.353233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:47.351090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:47.351090+0000","last_clean_scrub_stamp":"2026-02-24T18:40:47.351090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T21:23:28.596399+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:34.995215+0000","last_change":"2026-02-24T18:35:32.921734+0000","last_active":"2026-02-24T18:37:34.995215+0000","last_peered":"2026-02-24T18:37:34.995215+0000","last_clean":"2026-02-24T18:37:34.995215+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:37:34.995215+0000","last_undegraded":"2026-02-24T18:37:34.995215+0000","last_fullsized":"2026-02-24T18:37:34.995215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:18.790534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:37.965070+0000","last_change":"2026-02-24T18:35:32.494925+0000","last_active":"2026-02-24T18:37:37.965070+0000","last_peered":"2026-02-24T18:37:37.965070+0000","last_clean":"2026-02-24T18:37:37.965070+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:37:37.965070+0000","last_undegraded":"2026-02-24T18:37:37.965070+0000","last_fullsized":"2026-02-24T18:37:37.965070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:41.424787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:46.676439+0000","last_change":"2026-02-24T18:40:46.676796+0000","last_active":"2026-02-24T18:40:46.676439+0000","last_peered":"2026-02-24T18:40:46.676439+0000","last_clean":"2026-02-24T18:40:46.676439+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:40:46.676439+0000","last_undegraded":"2026-02-24T18:40:46.676439+0000","last_fullsized":"2026-02-24T18:40:46.676439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:46.674855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:46.674855+0000","last_clean_scrub_stamp":"2026-02-24T18:40:46.674855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:54:20.811515+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:37:32.962781+0000","last_change":"2026-02-24T18:35:32.500422+0000","last_active":"2026-02-24T18:37:32.962781+0000","last_peered":"2026-02-24T18:37:32.962781+0000","last_clean":"2026-02-24T18:37:32.962781+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:37:32.962781+0000","last_undegraded":"2026-02-24T18:37:32.962781+0000","last_fullsized":"2026-02-24T18:37:32.962781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:35:29.149134+0000","last_clean_scrub_stamp":"2026-02-24T18:35:29.149134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:12:20.555837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"29'2","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:44.693269+0000","last_change":"2026-02-24T18:40:44.698445+0000","last_active":"2026-02-24T18:40:44.693269+0000","last_peered":"2026-02-24T18:40:44.693269+0000","last_clean":"2026-02-24T18:40:44.693269+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:40:44.693269+0000","last_undegraded":"2026-02-24T18:40:44.693269+0000","last_fullsized":"2026-02-24T18:40:44.693269+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2026-02-24T18:40:44.690244+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2026-02-24T18:40:44.690244+0000","last_clean_scrub_stamp":"2026-02-24T18:40:44.690244+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:14:04.133598+0000","scrub_duration":276,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"29'34","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:45.927425+0000","last_change":"2026-02-24T18:40:45.933642+0000","last_active":"2026-02-24T18:40:45.927425+0000","last_peered":"2026-02-24T18:40:45.927425+0000","last_clean":"2026-02-24T18:40:45.927425+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:40:45.927425+0000","last_undegraded":"2026-02-24T18:40:45.927425+0000","last_fullsized":"2026-02-24T18:40:45.927425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'34","last_scrub_stamp":"2026-02-24T18:40:45.924722+0000","last_deep_scrub":"29'34","last_deep_scrub_stamp":"2026-02-24T18:40:45.924722+0000","last_clean_scrub_stamp":"2026-02-24T18:40:45.924722+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:02:58.012011+0000","scrub_duration":281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313284,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:40:50.671 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-24T18:35:29.149134+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=35, tm_sec=29, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=40, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:40:50.672 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-24T18:35:29.149134+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=35, tm_sec=29, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=40, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:40:50.672 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-24T18:35:29.149134+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=35, tm_sec=29, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=40, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:40:50.672 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-24T18:35:29.149134+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=35, tm_sec=29, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=40, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:40:50.672 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-24T18:35:29.149134+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=35, tm_sec=29, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=40, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:40:50.673 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T18:40:59.769 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:37.715 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.771 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:37.717 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.780 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:37.725 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.798 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:37.706 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.798 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:37.743 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.799 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:37.707 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.807 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:37.715 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.808 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:37.753 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:40:59.817 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:37.762 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:10.674 DEBUG:teuthology.orchestra.run.trial018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.742+0000 7efd90978640 10 monclient: get_monmap_and_config 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.742+0000 7efd90978640 10 monclient: build_initial_monmap 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.742+0000 7efd90978640 10 monclient: monmap: 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:41:10.743814+0000 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:41:10.743814+0000 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:41:10.744 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:41:10.745 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 1 Processor -- start 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 1 -- start start 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient: init 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient: _reopen_session rank -1 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient: _add_conns ranks=[2,0,1,5,3,4] 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efd8806ff20 addr v1:10.20.193.110:6789/0 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efd880c99a0 addr v1:10.20.193.18:6789/0 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efd8815a5e0 addr v1:10.20.193.32:6789/0 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 1 -- --> v1:10.20.193.18:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd880c7e90 con 0x7efd880c99a0 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd880c8170 con 0x7efd8815a5e0 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd880c8450 con 0x7efd8806ff20 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient(hunting): _renew_subs 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.743+0000 7efd90978640 10 monclient(hunting): authenticate will time out at 1814.424979s 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.744+0000 7efd8e6ed640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7efd8806ff20 0x7efd88159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:45144/0 (socket says 10.20.193.18:45144) 2026-02-24T18:41:10.746 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.744+0000 7efd8e6ed640 1 -- 10.20.193.18:0/1959267255 learned_addr learned my addr 10.20.193.18:0/1959267255 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555065068 0 0) 0x7efd880c8450 con 0x7efd8806ff20 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _negotiate 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): my global_id is 5297 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd78003620 con 0x7efd8806ff20 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.0 v1:10.20.193.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2428727420 0 0) 0x7efd880c7e90 con 0x7efd880c99a0 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _negotiate 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 10 monclient(hunting): my global_id is 5406 2026-02-24T18:41:10.747 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.745+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> v1:10.20.193.18:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd880c8450 con 0x7efd880c99a0 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.1 v1:10.20.193.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 266896999 0 0) 0x7efd880c8170 con 0x7efd8815a5e0 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 10 monclient(hunting): _negotiate 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 10 monclient(hunting): my global_id is 5320 2026-02-24T18:41:10.748 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.746+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> v1:10.20.193.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd880c7e90 con 0x7efd8815a5e0 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.2 v1:10.20.193.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2463403424 0 0) 0x7efd78003620 con 0x7efd8806ff20 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 >> v1:10.20.193.32:6789/0 conn(0x7efd8815a5e0 legacy=0x7efd880c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 >> v1:10.20.193.18:6789/0 conn(0x7efd880c99a0 legacy=0x7efd8814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _finish_hunting 0 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 monclient: found mon.noname-c-legacy 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.110:6789/0 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> v1:10.20.193.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880c8730 con 0x7efd8806ff20 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _finish_auth 0 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _check_auth_tickets 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.2 v1:10.20.193.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 373154985 0 0) 0x7efd80002f00 con 0x7efd8806ff20 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd90978640 5 monclient: authenticate success, global_id 5297 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd90978640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: dump: 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.749 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] but i'm connected to v1:10.20.193.110:6789/0, reconnecting 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _reopen_session rank -1 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 >> v1:10.20.193.110:6789/0 conn(0x7efd8806ff20 legacy=0x7efd88159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd78005090 0x7efd780038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): picked mon.c con 0x7efd78005090 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78003de0 0x7efd78004260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): picked mon.b con 0x7efd78003de0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 0x7efd7800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): picked mon.a con 0x7efd78009ab0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800c570 con 0x7efd78009ab0 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800c720 con 0x7efd78003de0 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800c920 con 0x7efd78005090 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): _renew_subs 2026-02-24T18:41:10.750 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd8d6eb640 10 monclient(hunting): _finish_auth 0 2026-02-24T18:41:10.751 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.747+0000 7efd90978640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 0x7efd7800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8deec640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78003de0 0x7efd78004260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8e6ed640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd78005090 0x7efd780038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): get_auth_request con 0x7efd78009ab0 auth_method 0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8deec640 10 monclient(hunting): get_auth_request con 0x7efd78003de0 auth_method 0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8deec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8deec640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8deec640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.748+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): get_auth_request con 0x7efd78005090 auth_method 0 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.752 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 10 monclient(hunting): handle_auth_done global_id 5297 payload 293 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 -- 10.20.193.18:0/1959267255 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd78005090 msgr2=0x7efd780038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd78005090 0x7efd780038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 -- 10.20.193.18:0/1959267255 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78003de0 msgr2=0x7efd78004260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78003de0 0x7efd78004260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 10 monclient: _finish_hunting 0 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 monclient: found mon.a 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.18:3300/0 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 1 -- 10.20.193.18:0/1959267255 --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd78003620 con 0x7efd78009ab0 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 10 monclient: _finish_auth 0 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 10 monclient: _check_auth_tickets 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.749+0000 7efd8eeee640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.750+0000 7efd8eeee640 1 -- 10.20.193.18:0/1959267255 reap_dead start 2026-02-24T18:41:10.753 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.750+0000 7efd8eeee640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 0x7efd7800bea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c320 tx=0x7efd7c02f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67d399fb4b3052a server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.0 v2:10.20.193.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003860 con 0x7efd78009ab0 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: dump: 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:10.754 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.0 v2:10.20.193.18:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c003c70 con 0x7efd78009ab0 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: handle_config config(0 keys) 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 10 monclient: get_monmap_and_config success 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 1 -- 10.20.193.18:0/1959267255 <== mon.0 v2:10.20.193.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c005310 con 0x7efd78009ab0 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 4 set_mon_vals no callback set 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd8d6eb640 10 monclient: dump: 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.755 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 10 monclient: shutdown 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 -- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 msgr2=0x7efd7800bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 0x7efd7800bea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c320 tx=0x7efd7c02f610 comp rx=0 tx=0).stop 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 -- 10.20.193.18:0/1959267255 shutdown_connections 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 --2- 10.20.193.18:0/1959267255 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd78009ab0 0x7efd7800bea0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 -- 10.20.193.18:0/1959267255 >> 10.20.193.18:0/1959267255 conn(0x7efd880cd780 msgr2=0x7efd880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 -- 10.20.193.18:0/1959267255 shutdown_connections 2026-02-24T18:41:10.756 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.752+0000 7efd90978640 1 -- 10.20.193.18:0/1959267255 wait complete. 2026-02-24T18:41:10.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 10 monclient: build_initial_monmap 2026-02-24T18:41:10.757 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 10 monclient: monmap: 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 librados: starting msgr at 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 librados: starting objecter 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 Processor -- start 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 -- start start 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 librados: setting wanted keys 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 1 librados: calling monclient init 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.753+0000 7efd90978640 10 monclient: init 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient: _reopen_session rank -1 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78009ab0 0x7efd880c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): picked mon.noname-b con 0x7efd78009ab0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd880c10d0 0x7efd8806d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.758 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): picked mon.noname-a con 0x7efd880c10d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): picked mon.noname-c con 0x7efd8814e4a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7efd88070300 con 0x7efd880c10d0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd88159a40 con 0x7efd78009ab0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd88070480 con 0x7efd8814e4a0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): _renew_subs 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd90978640 10 monclient(hunting): authenticate will time out at 1814.435157s 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8e6ed640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78009ab0 0x7efd880c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd880c10d0 0x7efd8806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.110:3300/0 says I am v2:10.20.193.18:56730/0 (socket says 10.20.193.18:56730) 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 1 -- 10.20.193.18:0/362484043 learned_addr learned my addr 10.20.193.18:0/362484043 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): get_auth_request con 0x7efd8814e4a0 auth_method 0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): get_auth_request con 0x7efd880c10d0 auth_method 0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8e6ed640 10 monclient(hunting): get_auth_request con 0x7efd78009ab0 auth_method 0 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8e6ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:10.759 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8e6ed640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8e6ed640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8deec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.754+0000 7efd8eeee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.755+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.755+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.755+0000 7efd8e6ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 10 monclient(hunting): handle_auth_done global_id 5300 payload 824 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 -- 10.20.193.18:0/362484043 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78009ab0 msgr2=0x7efd880c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78009ab0 0x7efd880c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 -- 10.20.193.18:0/362484043 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd880c10d0 msgr2=0x7efd8806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd880c10d0 0x7efd8806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 10 monclient: _finish_hunting 0 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 monclient: found mon.noname-c 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.110:3300/0 2026-02-24T18:41:10.760 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.756+0000 7efd8eeee640 1 -- 10.20.193.18:0/362484043 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880c1600 con 0x7efd8814e4a0 2026-02-24T18:41:10.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.757+0000 7efd8eeee640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c450 tx=0x7efd7c002e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:10.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.757+0000 7efd72ffd640 10 client.?.objecter ms_handle_connect 0x7efd8814e4a0 2026-02-24T18:41:10.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.757+0000 7efd72ffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:41:10.761 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.758+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c046070 con 0x7efd8814e4a0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.758+0000 7efd72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.758+0000 7efd72ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.758+0000 7efd72ffd640 10 monclient: dump: 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: _finish_auth 0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: _check_auth_tickets 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 5 monclient: authenticate success, global_id 5300 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _renew_subs 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c036610 con 0x7efd8814e4a0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd880c3010 con 0x7efd8814e4a0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003860 con 0x7efd8814e4a0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:10.762 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd72ffd640 10 monclient: dump: 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd8ceea640 4 set_mon_vals no callback set 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 client.5300.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _renew_subs 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd880c35d0 con 0x7efd8814e4a0 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 1 librados: init done 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.110:3300/0 2026-02-24T18:41:10.763 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.759+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd88159ed0 con 0x7efd8814e4a0 2026-02-24T18:41:10.771 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.770+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efd7c0369c0 con 0x7efd8814e4a0 2026-02-24T18:41:10.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.771+0000 7efd72ffd640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 0x7efd5807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:10.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.771+0000 7efd8e6ed640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 0x7efd5807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:10.772 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.771+0000 7efd8e6ed640 10 monclient: get_auth_request con 0x7efd58078e00 auth_method 0 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.771+0000 7efd72ffd640 10 client.5300.objecter ms_handle_connect 0x7efd58078e00 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.771+0000 7efd8e6ed640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 0x7efd5807b2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd80002a50 tx=0x7efd800035a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.772+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7efd7c036ce0 con 0x7efd8814e4a0 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.772+0000 7efd72ffd640 10 client.5300.objecter ms_dispatch2 0x7efd88000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.772+0000 7efd72ffd640 3 client.5300.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.772+0000 7efd72ffd640 3 client.5300.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:41:10.773 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.772+0000 7efd72ffd640 20 client.5300.objecter dump_active .. 0 homeless 2026-02-24T18:41:10.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.870+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mon.2 v2:10.20.193.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efd7c003e20 con 0x7efd8814e4a0 2026-02-24T18:41:10.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.870+0000 7efd72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:10.872 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.870+0000 7efd72ffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:41:10.953 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.952+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 --> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd8814e920 con 0x7efd58078e00 2026-02-24T18:41:10.954 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.953+0000 7efd72ffd640 1 -- 10.20.193.18:0/362484043 <== mgr.4104 v2:10.20.193.112:6800/3643511854 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+44625 (secure 0 0 0) 0x7efd80004d80 con 0x7efd58078e00 2026-02-24T18:41:10.955 INFO:teuthology.orchestra.run.trial018.stdout: 2026-02-24T18:41:10.955 INFO:teuthology.orchestra.run.trial018.stderr:dumped all 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 10 librados: watch_flush enter 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 10 librados: watch_flush exit 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 20 client.5300.objecter shutdown clearing up homeless session... 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 10 client.5300.objecter successfully canceled tick 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 msgr2=0x7efd5807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 0x7efd5807b2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd80002a50 tx=0x7efd800035a0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 10 monclient: shutdown 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 msgr2=0x7efd881f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c450 tx=0x7efd7c002e40 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 shutdown_connections 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7efd58078e00 0x7efd5807b2c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7efd8814e4a0 0x7efd881f4040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7efd880c10d0 0x7efd8806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 --2- 10.20.193.18:0/362484043 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7efd78009ab0 0x7efd880c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 >> 10.20.193.18:0/362484043 conn(0x7efd880cd780 msgr2=0x7efd88158070 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 shutdown_connections 2026-02-24T18:41:10.956 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 -- 10.20.193.18:0/362484043 wait complete. 2026-02-24T18:41:10.957 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:10.954+0000 7efd90978640 1 librados: shutdown 2026-02-24T18:41:10.964 INFO:teuthology.orchestra.run.trial018.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2026-02-24T18:41:09.861233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437108,"kb_used_data":5720,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512332,"statfs":{"total":1546188226560,"available":1545740627968,"internally_reserved":0,"allocated":5857280,"data_stored":2838502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25506,"internal_metadata":441424990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001871"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:51.394948+0000","last_change":"2026-02-24T18:40:51.397625+0000","last_active":"2026-02-24T18:40:51.394948+0000","last_peered":"2026-02-24T18:40:51.394948+0000","last_clean":"2026-02-24T18:40:51.394948+0000","last_became_active":"2026-02-24T18:35:30.672400+0000","last_became_peered":"2026-02-24T18:35:30.672400+0000","last_unstale":"2026-02-24T18:40:51.394948+0000","last_undegraded":"2026-02-24T18:40:51.394948+0000","last_fullsized":"2026-02-24T18:40:51.394948+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:51.391793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:51.391793+0000","last_clean_scrub_stamp":"2026-02-24T18:40:51.391793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:06.758565+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0.036849057999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:49.660672+0000","last_change":"2026-02-24T18:40:49.665342+0000","last_active":"2026-02-24T18:40:49.660672+0000","last_peered":"2026-02-24T18:40:49.660672+0000","last_clean":"2026-02-24T18:40:49.660672+0000","last_became_active":"2026-02-24T18:35:30.712890+0000","last_became_peered":"2026-02-24T18:35:30.712890+0000","last_unstale":"2026-02-24T18:40:49.660672+0000","last_undegraded":"2026-02-24T18:40:49.660672+0000","last_fullsized":"2026-02-24T18:40:49.660672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:49.658482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:49.658482+0000","last_clean_scrub_stamp":"2026-02-24T18:40:49.658482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T19:58:25.491368+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":0.032722372999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:47.353233+0000","last_change":"2026-02-24T18:40:47.357852+0000","last_active":"2026-02-24T18:40:47.353233+0000","last_peered":"2026-02-24T18:40:47.353233+0000","last_clean":"2026-02-24T18:40:47.353233+0000","last_became_active":"2026-02-24T18:35:30.702774+0000","last_became_peered":"2026-02-24T18:35:30.702774+0000","last_unstale":"2026-02-24T18:40:47.353233+0000","last_undegraded":"2026-02-24T18:40:47.353233+0000","last_fullsized":"2026-02-24T18:40:47.353233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:47.351090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:47.351090+0000","last_clean_scrub_stamp":"2026-02-24T18:40:47.351090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T21:23:28.596399+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0.038697336999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:48.499923+0000","last_change":"2026-02-24T18:40:48.504662+0000","last_active":"2026-02-24T18:40:48.499923+0000","last_peered":"2026-02-24T18:40:48.499923+0000","last_clean":"2026-02-24T18:40:48.499923+0000","last_became_active":"2026-02-24T18:35:30.722185+0000","last_became_peered":"2026-02-24T18:35:30.722185+0000","last_unstale":"2026-02-24T18:40:48.499923+0000","last_undegraded":"2026-02-24T18:40:48.499923+0000","last_fullsized":"2026-02-24T18:40:48.499923+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:48.493168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:48.493168+0000","last_clean_scrub_stamp":"2026-02-24T18:40:48.493168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:43:53.240991+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":0.032717414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"29'3","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:50.434492+0000","last_change":"2026-02-24T18:40:50.435338+0000","last_active":"2026-02-24T18:40:50.434492+0000","last_peered":"2026-02-24T18:40:50.434492+0000","last_clean":"2026-02-24T18:40:50.434492+0000","last_became_active":"2026-02-24T18:35:30.715167+0000","last_became_peered":"2026-02-24T18:35:30.715167+0000","last_unstale":"2026-02-24T18:40:50.434492+0000","last_undegraded":"2026-02-24T18:40:50.434492+0000","last_fullsized":"2026-02-24T18:40:50.434492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2026-02-24T18:40:50.433082+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2026-02-24T18:40:50.433082+0000","last_clean_scrub_stamp":"2026-02-24T18:40:50.433082+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:01:19.741271+0000","scrub_duration":174,"objects_trimmed":0,"snaptrim_duration":0.054788006,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:46.676439+0000","last_change":"2026-02-24T18:40:46.676796+0000","last_active":"2026-02-24T18:40:46.676439+0000","last_peered":"2026-02-24T18:40:46.676439+0000","last_clean":"2026-02-24T18:40:46.676439+0000","last_became_active":"2026-02-24T18:35:30.570325+0000","last_became_peered":"2026-02-24T18:35:30.570325+0000","last_unstale":"2026-02-24T18:40:46.676439+0000","last_undegraded":"2026-02-24T18:40:46.676439+0000","last_fullsized":"2026-02-24T18:40:46.676439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:46.674855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:46.674855+0000","last_clean_scrub_stamp":"2026-02-24T18:40:46.674855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T20:54:20.811515+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0.037250737999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:49.385819+0000","last_change":"2026-02-24T18:40:49.390518+0000","last_active":"2026-02-24T18:40:49.385819+0000","last_peered":"2026-02-24T18:40:49.385819+0000","last_clean":"2026-02-24T18:40:49.385819+0000","last_became_active":"2026-02-24T18:35:30.559556+0000","last_became_peered":"2026-02-24T18:35:30.559556+0000","last_unstale":"2026-02-24T18:40:49.385819+0000","last_undegraded":"2026-02-24T18:40:49.385819+0000","last_fullsized":"2026-02-24T18:40:49.385819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:40:49.383658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:40:49.383658+0000","last_clean_scrub_stamp":"2026-02-24T18:40:49.383658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:49:24.221149+0000","scrub_duration":115,"objects_trimmed":0,"snaptrim_duration":0.055529027000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"29'2","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:44.693269+0000","last_change":"2026-02-24T18:40:44.698445+0000","last_active":"2026-02-24T18:40:44.693269+0000","last_peered":"2026-02-24T18:40:44.693269+0000","last_clean":"2026-02-24T18:40:44.693269+0000","last_became_active":"2026-02-24T18:35:30.593276+0000","last_became_peered":"2026-02-24T18:35:30.593276+0000","last_unstale":"2026-02-24T18:40:44.693269+0000","last_undegraded":"2026-02-24T18:40:44.693269+0000","last_fullsized":"2026-02-24T18:40:44.693269+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2026-02-24T18:40:44.690244+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2026-02-24T18:40:44.690244+0000","last_clean_scrub_stamp":"2026-02-24T18:40:44.690244+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:14:04.133598+0000","scrub_duration":276,"objects_trimmed":0,"snaptrim_duration":0.030867374,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"29'34","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2026-02-24T18:40:45.927425+0000","last_change":"2026-02-24T18:40:45.933642+0000","last_active":"2026-02-24T18:40:45.927425+0000","last_peered":"2026-02-24T18:40:45.927425+0000","last_clean":"2026-02-24T18:40:45.927425+0000","last_became_active":"2026-02-24T18:35:28.827601+0000","last_became_peered":"2026-02-24T18:35:28.827601+0000","last_unstale":"2026-02-24T18:40:45.927425+0000","last_undegraded":"2026-02-24T18:40:45.927425+0000","last_fullsized":"2026-02-24T18:40:45.927425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'34","last_scrub_stamp":"2026-02-24T18:40:45.924722+0000","last_deep_scrub":"29'34","last_deep_scrub_stamp":"2026-02-24T18:40:45.924722+0000","last_clean_scrub_stamp":"2026-02-24T18:40:45.924722+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:02:58.012011+0000","scrub_duration":281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":23,"seq":98784247879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":23,"seq":98784247880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313288,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,5,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,4,5,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313287,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,2,4,6,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5,6,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280583,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[2,4,5,6,7,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":4,"up_from":22,"seq":94489280583,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,5,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":22,"seq":94489280583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[1,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":319488,"data_stored":127693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 5 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":23,"seq":98784247879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[0,2,5,6,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":8,"up_from":23,"seq":98784247879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":716800,"data_stored":525514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[4,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":23,"seq":98784247879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":315392,"data_stored":127674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:41:10.965 DEBUG:teuthology.orchestra.run.trial018:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.031+0000 7f56e88b0640 10 monclient: get_monmap_and_config 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.031+0000 7f56e88b0640 10 monclient: build_initial_monmap 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.031+0000 7f56e88b0640 10 monclient: monmap: 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:41:11.032569+0000 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:41:11.032569+0000 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:0: v1:10.20.193.18:6789/0 mon.noname-a-legacy 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:1: v1:10.20.193.32:6789/0 mon.noname-b-legacy 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:2: v1:10.20.193.110:6789/0 mon.noname-c-legacy 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:3: v2:10.20.193.18:3300/0 mon.noname-a 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:4: v2:10.20.193.32:3300/0 mon.noname-b 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr:5: v2:10.20.193.110:3300/0 mon.noname-c 2026-02-24T18:41:11.033 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 Processor -- start 2026-02-24T18:41:11.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 -- start start 2026-02-24T18:41:11.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient: init 2026-02-24T18:41:11.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:41:11.034 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient: _add_conns ranks=[2,4,1,3,0,5] 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f56e01527d0 addr v1:10.20.193.110:6789/0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f56e0152bb0 0x7f56e00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-b con 0x7f56e0152bb0 addr v2:10.20.193.32:3300/0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f56e00c5fc0 addr v1:10.20.193.32:6789/0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 -- --> v1:10.20.193.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e006e300 con 0x7f56e00c5fc0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 -- --> v1:10.20.193.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e006e5e0 con 0x7f56e01527d0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 1 -- --> v2:10.20.193.32:3300/0 -- mon_getmap magic: 0 -- 0x7f56e006f1f0 con 0x7f56e0152bb0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.032+0000 7f56e88b0640 10 monclient(hunting): authenticate will time out at 1814.713726s 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e6625640 1 --1- >> v1:10.20.193.110:6789/0 conn(0x7f56e01527d0 0x7f56e006da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.110:6789/0 says I am v1:10.20.193.18:45148/0 (socket says 10.20.193.18:45148) 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e6625640 1 -- 10.20.193.18:0/1504997193 learned_addr learned my addr 10.20.193.18:0/1504997193 (peer_addr_for_me v1:10.20.193.18:0/0) 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 1 --2- >> v2:10.20.193.32:3300/0 conn(0x7f56e0152bb0 0x7f56e00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): get_auth_request con 0x7f56e0152bb0 auth_method 0 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:11.035 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.033+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 1 -- 10.20.193.18:0/1504997193 <== mon.2 v1:10.20.193.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675993429 0 0) 0x7f56e006e5e0 con 0x7f56e01527d0 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 10 monclient(hunting): _negotiate 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 10 monclient(hunting): my global_id is 5303 2026-02-24T18:41:11.036 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.034+0000 7f56e5623640 1 -- 10.20.193.18:0/1504997193 --> v1:10.20.193.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56d0003620 con 0x7f56e01527d0 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 10 monclient(hunting): handle_auth_done global_id 5326 payload 274 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 1 -- 10.20.193.18:0/1504997193 >> v1:10.20.193.110:6789/0 conn(0x7f56e01527d0 legacy=0x7f56e006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 1 -- 10.20.193.18:0/1504997193 >> v1:10.20.193.32:6789/0 conn(0x7f56e00c5fc0 legacy=0x7f56e00c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 10 monclient: _finish_hunting 0 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 1 monclient: found mon.noname-b 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.037 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.035+0000 7f56e5e24640 1 -- 10.20.193.18:0/1504997193 --> v2:10.20.193.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e006e8c0 con 0x7f56e0152bb0 2026-02-24T18:41:11.038 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.037+0000 7f56e5e24640 1 --2- 10.20.193.18:0/1504997193 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e00c5a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0098a0 tx=0x7f56cc02ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2df75374da9ed0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:11.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 1 -- 10.20.193.18:0/1504997193 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc03c070 con 0x7f56e0152bb0 2026-02-24T18:41:11.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:11.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:41:11.040 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: dump: 2026-02-24T18:41:11.040 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: _finish_auth 0 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: _check_auth_tickets 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 1 -- 10.20.193.18:0/1504997193 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc02fb70 con 0x7f56e0152bb0 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: handle_config config(0 keys) 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 1 -- 10.20.193.18:0/1504997193 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc037440 con 0x7f56e0152bb0 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e88b0640 5 monclient: authenticate success, global_id 5326 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e88b0640 10 monclient: get_monmap_and_config success 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e88b0640 4 set_mon_vals no callback set 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.038+0000 7f56e5623640 10 monclient: dump: 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:11.041 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 10 monclient: shutdown 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 -- 10.20.193.18:0/1504997193 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 msgr2=0x7f56e00c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1504997193 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e00c5a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0098a0 tx=0x7f56cc02ee60 comp rx=0 tx=0).stop 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 -- 10.20.193.18:0/1504997193 shutdown_connections 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1504997193 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e00c5a80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 -- 10.20.193.18:0/1504997193 >> 10.20.193.18:0/1504997193 conn(0x7f56e00cd780 msgr2=0x7f56e00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 -- 10.20.193.18:0/1504997193 shutdown_connections 2026-02-24T18:41:11.042 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 1 -- 10.20.193.18:0/1504997193 wait complete. 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 10 monclient: build_initial_monmap 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.039+0000 7f56e88b0640 10 monclient: monmap: 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:epoch 0 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 0.000000 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:created 0.000000 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 0 (unknown) 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.noname-a 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.noname-b 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-c 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 librados: starting msgr at 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 librados: starting objecter 2026-02-24T18:41:11.044 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 Processor -- start 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 -- start start 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 librados: setting wanted keys 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 librados: calling monclient init 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient: init 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient: _reopen_session rank -1 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56e00c5fc0 0x7f56e0073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-c con 0x7f56e00c5fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 0x7f56e01482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-a con 0x7f56e01527d0 addr [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e01f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): picked mon.noname-b con 0x7f56e0152bb0 addr [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 -- --> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56e006ee80 con 0x7f56e01527d0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 -- --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56e00c6470 con 0x7f56e0152bb0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): start opening mon connection 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 1 -- --> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56e00c9570 con 0x7f56e00c5fc0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): _renew_subs 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e88b0640 10 monclient(hunting): authenticate will time out at 1814.721669s 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e5e24640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 0x7f56e01482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.040+0000 7f56e6e26640 1 --2- >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e01f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 1 --2- >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56e00c5fc0 0x7f56e0073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 1 --2- >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 0x7f56e01482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.18:3300/0 says I am v2:10.20.193.18:40718/0 (socket says 10.20.193.18:40718) 2026-02-24T18:41:11.045 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 1 -- 10.20.193.18:0/1353233291 learned_addr learned my addr 10.20.193.18:0/1353233291 (peer_addr_for_me v2:10.20.193.18:0/0) 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): get_auth_request con 0x7f56e01527d0 auth_method 0 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): get_auth_request con 0x7f56e0152bb0 auth_method 0 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): get_auth_request con 0x7f56e00c5fc0 auth_method 0 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): _init_auth method 2 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6e26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e5e24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.041+0000 7f56e6625640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 10 monclient(hunting): handle_auth_done global_id 5332 payload 824 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 -- 10.20.193.18:0/1353233291 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56e00c5fc0 msgr2=0x7f56e0073500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56e00c5fc0 0x7f56e0073500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.046 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 -- 10.20.193.18:0/1353233291 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 msgr2=0x7f56e01482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 0x7f56e01482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 10 monclient: _finish_hunting 0 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 monclient: found mon.noname-b 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.042+0000 7f56e6e26640 1 -- 10.20.193.18:0/1353233291 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e0073c20 con 0x7f56e0152bb0 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.044+0000 7f56e6e26640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e01f4d70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56d400cbc0 tx=0x7f56d400a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.044+0000 7f56caffd640 10 client.?.objecter ms_handle_connect 0x7f56e0152bb0 2026-02-24T18:41:11.047 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.044+0000 7f56caffd640 10 client.?.objecter resend_mon_ops 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400ad50 con 0x7f56e0152bb0 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: dump: 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.048 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: _finish_auth 0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: _check_auth_tickets 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d4003a30 con 0x7f56e0152bb0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 5 monclient: authenticate success, global_id 5332 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 10 monclient: _renew_subs 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e0073f00 con 0x7f56e0152bb0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: handle_config config(0 keys) 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4003e40 con 0x7f56e0152bb0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e4e22640 4 set_mon_vals no callback set 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56caffd640 10 monclient: dump: 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:epoch 1 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:fsid 8c4cd187-4cf3-4491-88ba-bf63c2f42ea1 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:last_changed 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:created 2026-02-24T18:32:00.393091+0000 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:min_mon_release 19 (squid) 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:election_strategy: 1 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:0: [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] mon.a 2026-02-24T18:41:11.049 INFO:teuthology.orchestra.run.trial018.stderr:1: [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] mon.b 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.c 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 10 client.5332.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 10 monclient: _renew_subs 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e00744c0 con 0x7f56e0152bb0 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.046+0000 7f56e88b0640 1 librados: init done 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.047+0000 7f56e88b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.047+0000 7f56e88b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.047+0000 7f56e88b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.050 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.047+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e00c5a80 con 0x7f56e0152bb0 2026-02-24T18:41:11.154 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.153+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f56d40052e0 con 0x7f56e0152bb0 2026-02-24T18:41:11.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.154+0000 7f56caffd640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 0x7f56b007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-24T18:41:11.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.154+0000 7f56e6625640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 0x7f56b007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-24T18:41:11.156 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.154+0000 7f56e6625640 10 monclient: get_auth_request con 0x7f56b0078e50 auth_method 0 2026-02-24T18:41:11.157 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 8944+0+0 (secure 0 0 0) 0x7f56d405e880 con 0x7f56e0152bb0 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 10 client.5332.objecter ms_dispatch2 0x7f56e0000b90 osd_map(29..29 src has 1..29) 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 3 client.5332.objecter handle_osd_map got epochs [29,29] > 0 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 3 client.5332.objecter handle_osd_map decoding full epoch 29 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 20 client.5332.objecter dump_active .. 0 homeless 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56e6625640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 0x7f56b007b310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005980 tx=0x7f56d8007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-24T18:41:11.158 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.155+0000 7f56caffd640 10 client.5332.objecter ms_handle_connect 0x7f56b0078e50 2026-02-24T18:41:11.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.161+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f56d4062840 con 0x7f56e0152bb0 2026-02-24T18:41:11.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.161+0000 7f56caffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-24T18:41:11.163 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.161+0000 7f56caffd640 10 monclient: _finish_command 1 = system:0 2026-02-24T18:41:11.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.243+0000 7f56e88b0640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-24T18:41:11.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.243+0000 7f56e88b0640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-24T18:41:11.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.243+0000 7f56e88b0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.32:3300/0 2026-02-24T18:41:11.245 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.243+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 --> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f56e0146650 con 0x7f56e0152bb0 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f56d4066b40 con 0x7f56e0152bb0 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56caffd640 10 monclient: handle_config config(1 keys) 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56e4e22640 4 set_mon_vals no callback set 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56e4e22640 10 set_mon_vals mon_health_to_clog = false 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56caffd640 1 -- 10.20.193.18:0/1353233291 <== mon.1 v2:10.20.193.32:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (secure 0 0 0) 0x7f56d406b9f0 con 0x7f56e0152bb0 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56caffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-24T18:41:11.341 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.339+0000 7f56caffd640 10 monclient: _finish_command 2 = system:0 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 10 librados: watch_flush enter 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 10 librados: watch_flush exit 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 20 client.5332.objecter shutdown clearing up homeless session... 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 10 client.5332.objecter successfully canceled tick 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 msgr2=0x7f56b007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 0x7f56b007b310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005980 tx=0x7f56d8007dd0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 10 monclient: shutdown 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 msgr2=0x7f56e01f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e01f4d70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56d400cbc0 tx=0x7f56d400a430 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 shutdown_connections 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.112:6800/3643511854,v1:10.20.193.112:6801/3643511854] conn(0x7f56b0078e50 0x7f56b007b310 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.32:3300/0,v1:10.20.193.32:6789/0] conn(0x7f56e0152bb0 0x7f56e01f4d70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.18:3300/0,v1:10.20.193.18:6789/0] conn(0x7f56e01527d0 0x7f56e01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 --2- 10.20.193.18:0/1353233291 >> [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] conn(0x7f56e00c5fc0 0x7f56e0073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 >> 10.20.193.18:0/1353233291 conn(0x7f56e00cd780 msgr2=0x7f56e00ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 shutdown_connections 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 -- 10.20.193.18:0/1353233291 wait complete. 2026-02-24T18:41:11.343 INFO:teuthology.orchestra.run.trial018.stderr:2026-02-24T18:41:11.341+0000 7f56e88b0640 1 librados: shutdown 2026-02-24T18:41:11.353 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-24T18:41:11.354 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-24T18:41:11.355 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-24T18:41:11.355 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-24T18:41:11.355 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:11.362 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:41:11.356+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 45695) UID: 0 2026-02-24T18:41:11.362 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:41:11.357+0000 beae640 -1 osd.0 29 *** Got signal Terminated *** 2026-02-24T18:41:12.587 INFO:tasks.ceph.osd.0.trial018.stderr:2026-02-24T18:41:12.584+0000 a699640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-24T18:41:16.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-24T18:41:17.456 INFO:tasks.ceph.osd.0:Stopped 2026-02-24T18:41:17.457 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-24T18:41:17.457 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:17.464 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:41:17.458+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 45696) UID: 0 2026-02-24T18:41:17.464 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:41:17.458+0000 beae640 -1 osd.4 31 *** Got signal Terminated *** 2026-02-24T18:41:18.085 INFO:tasks.ceph.osd.4.trial018.stderr:2026-02-24T18:41:18.082+0000 c6af640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-24T18:41:19.771 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:57.716 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.797 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:57.743 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.807 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:07:57.753 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.824 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:57.770 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.825 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:57.733 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.826 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:57.734 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.835 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:57.780 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.845 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:07:57.790 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:19.846 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:07:57.754 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:23.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2026-02-24T18:41:23.091 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-24T18:41:23.559 INFO:tasks.ceph.osd.4:Stopped 2026-02-24T18:41:23.559 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-24T18:41:23.559 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:23.567 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:41:23.561+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 (PID: 45700) UID: 0 2026-02-24T18:41:23.567 INFO:tasks.ceph.osd.8.trial018.stderr:2026-02-24T18:41:23.561+0000 beae640 -1 osd.8 33 *** Got signal Terminated *** 2026-02-24T18:41:29.661 INFO:tasks.ceph.osd.8:Stopped 2026-02-24T18:41:29.661 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2026-02-24T18:41:29.661 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:29.669 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:41:29.663+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 (PID: 45702) UID: 0 2026-02-24T18:41:29.669 INFO:tasks.ceph.osd.12.trial018.stderr:2026-02-24T18:41:29.663+0000 beae640 -1 osd.12 35 *** Got signal Terminated *** 2026-02-24T18:41:29.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 is failed for ~0s 2026-02-24T18:41:29.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2026-02-24T18:41:35.763 INFO:tasks.ceph.osd.12:Stopped 2026-02-24T18:41:35.763 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-24T18:41:35.763 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:35.770 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:41:35.763+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 44806) UID: 0 2026-02-24T18:41:35.771 INFO:tasks.ceph.osd.1.trial032.stderr:2026-02-24T18:41:35.764+0000 beae640 -1 osd.1 36 *** Got signal Terminated *** 2026-02-24T18:41:36.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.12 is failed for ~0s 2026-02-24T18:41:36.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 has been restored 2026-02-24T18:41:39.825 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:17.771 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.828 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:17.774 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.829 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:17.774 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.838 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:17.784 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.844 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:17.790 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.851 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:17.796 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.855 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:17.763 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.857 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:17.765 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:39.867 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:17.776 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:41.865 INFO:tasks.ceph.osd.1:Stopped 2026-02-24T18:41:41.865 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-24T18:41:41.865 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:41.872 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:41:41.865+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 44809) UID: 0 2026-02-24T18:41:41.873 INFO:tasks.ceph.osd.5.trial032.stderr:2026-02-24T18:41:41.866+0000 beae640 -1 osd.5 38 *** Got signal Terminated *** 2026-02-24T18:41:43.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-24T18:41:43.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.12 has been restored 2026-02-24T18:41:47.966 INFO:tasks.ceph.osd.5:Stopped 2026-02-24T18:41:47.967 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-24T18:41:47.967 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:47.975 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:41:47.967+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 (PID: 44811) UID: 0 2026-02-24T18:41:47.975 INFO:tasks.ceph.osd.9.trial032.stderr:2026-02-24T18:41:47.968+0000 beae640 -1 osd.9 40 *** Got signal Terminated *** 2026-02-24T18:41:49.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2026-02-24T18:41:54.069 INFO:tasks.ceph.osd.9:Stopped 2026-02-24T18:41:54.070 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2026-02-24T18:41:54.070 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:41:54.077 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:41:54.070+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 (PID: 44814) UID: 0 2026-02-24T18:41:54.078 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:41:54.071+0000 beae640 -1 osd.13 42 *** Got signal Terminated *** 2026-02-24T18:41:54.564 INFO:tasks.ceph.osd.13.trial032.stderr:2026-02-24T18:41:54.561+0000 c6af640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-24T18:41:59.847 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:37.793 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.872 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:37.818 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.882 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:37.828 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.898 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:37.844 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.899 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:37.845 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.900 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:37.809 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.901 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:37.809 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.910 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:37.855 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:41:59.912 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:37.820 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:00.172 INFO:tasks.ceph.osd.13:Stopped 2026-02-24T18:42:00.172 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-24T18:42:00.172 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:00.180 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:42:00.174+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 45403) UID: 0 2026-02-24T18:42:00.181 INFO:tasks.ceph.osd.2.trial110.stderr:2026-02-24T18:42:00.175+0000 beae640 -1 osd.2 44 *** Got signal Terminated *** 2026-02-24T18:42:06.275 INFO:tasks.ceph.osd.2:Stopped 2026-02-24T18:42:06.275 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-24T18:42:06.275 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:06.283 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:42:06.277+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 45406) UID: 0 2026-02-24T18:42:06.283 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T18:42:06.278+0000 beae640 -1 osd.6 45 *** Got signal Terminated *** 2026-02-24T18:42:12.376 INFO:tasks.ceph.osd.6:Stopped 2026-02-24T18:42:12.377 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2026-02-24T18:42:12.377 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:12.384 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:42:12.379+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 (PID: 45409) UID: 0 2026-02-24T18:42:12.384 INFO:tasks.ceph.osd.10.trial110.stderr:2026-02-24T18:42:12.380+0000 beae640 -1 osd.10 46 *** Got signal Terminated *** 2026-02-24T18:42:18.478 INFO:tasks.ceph.osd.10:Stopped 2026-02-24T18:42:18.479 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2026-02-24T18:42:18.479 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:18.486 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:42:18.481+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 (PID: 45412) UID: 0 2026-02-24T18:42:18.486 INFO:tasks.ceph.osd.14.trial110.stderr:2026-02-24T18:42:18.482+0000 beae640 -1 osd.14 48 *** Got signal Terminated *** 2026-02-24T18:42:19.894 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:57.839 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.895 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:57.841 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.905 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:08:57.850 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.919 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:57.865 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.921 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:57.866 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.924 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:57.832 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.925 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:57.833 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.931 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:08:57.877 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:19.934 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:08:57.842 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:24.581 INFO:tasks.ceph.osd.14:Stopped 2026-02-24T18:42:24.582 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-24T18:42:24.582 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:24.600 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:42:24.592+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 45515) UID: 0 2026-02-24T18:42:24.601 INFO:tasks.ceph.osd.3.trial112.stderr:2026-02-24T18:42:24.593+0000 beae640 -1 osd.3 49 *** Got signal Terminated *** 2026-02-24T18:42:30.683 INFO:tasks.ceph.osd.3:Stopped 2026-02-24T18:42:30.684 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-24T18:42:30.684 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:30.692 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:42:30.684+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 45520) UID: 0 2026-02-24T18:42:30.692 INFO:tasks.ceph.osd.7.trial112.stderr:2026-02-24T18:42:30.684+0000 beae640 -1 osd.7 49 *** Got signal Terminated *** 2026-02-24T18:42:36.786 INFO:tasks.ceph.osd.7:Stopped 2026-02-24T18:42:36.786 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2026-02-24T18:42:36.786 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:36.795 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:42:36.787+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 (PID: 45524) UID: 0 2026-02-24T18:42:36.795 INFO:tasks.ceph.osd.11.trial112.stderr:2026-02-24T18:42:36.788+0000 beae640 -1 osd.11 49 *** Got signal Terminated *** 2026-02-24T18:42:39.925 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:09:17.871 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.928 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:09:17.873 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.935 INFO:tasks.ceph.mon.a.trial018.stderr:==00:00:09:17.881 45353== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.952 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:09:17.898 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.953 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:09:17.860 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.953 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:09:17.898 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.953 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:09:17.861 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.960 INFO:tasks.ceph.mon.b.trial032.stderr:==00:00:09:17.905 44572== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:39.961 INFO:tasks.ceph.mon.c.trial110.stderr:==00:00:09:17.869 45171== Warning: unimplemented fcntl command: 1036 2026-02-24T18:42:42.887 INFO:tasks.ceph.osd.11:Stopped 2026-02-24T18:42:42.888 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2026-02-24T18:42:42.888 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:42.896 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:42:42.889+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 (PID: 45526) UID: 0 2026-02-24T18:42:42.896 INFO:tasks.ceph.osd.15.trial112.stderr:2026-02-24T18:42:42.889+0000 beae640 -1 osd.15 49 *** Got signal Terminated *** 2026-02-24T18:42:48.989 INFO:tasks.ceph.osd.15:Stopped 2026-02-24T18:42:48.990 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-24T18:42:48.991 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-24T18:42:48.992 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:49.023 INFO:tasks.ceph.mgr.y:Stopped 2026-02-24T18:42:49.023 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-24T18:42:49.023 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:49.024 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:42:49.021+0000 7f829e801640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 44454) UID: 0 2026-02-24T18:42:49.025 INFO:tasks.ceph.mgr.x.trial112.stderr:2026-02-24T18:42:49.021+0000 7f829e801640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-24T18:42:49.055 INFO:tasks.ceph.mgr.x:Stopped 2026-02-24T18:42:49.055 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-24T18:42:49.055 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-24T18:42:49.055 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:49.074 INFO:tasks.ceph.mon.a.trial018.stderr:2026-02-24T18:42:49.066+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 45351) UID: 0 2026-02-24T18:42:49.074 INFO:tasks.ceph.mon.a.trial018.stderr:2026-02-24T18:42:49.066+0000 aca6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-24T18:42:49.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2026-02-24T18:42:54.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-24T18:42:54.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2026-02-24T18:42:55.157 INFO:tasks.ceph.mon.a:Stopped 2026-02-24T18:42:55.158 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-24T18:42:55.158 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:42:55.166 INFO:tasks.ceph.mon.b.trial032.stderr:2026-02-24T18:42:55.157+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 44570) UID: 0 2026-02-24T18:42:55.166 INFO:tasks.ceph.mon.b.trial032.stderr:2026-02-24T18:42:55.158+0000 aca6640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2026-02-24T18:42:59.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2026-02-24T18:42:59.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-24T18:43:01.260 INFO:tasks.ceph.mon.b:Stopped 2026-02-24T18:43:01.261 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-24T18:43:01.261 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:43:01.268 INFO:tasks.ceph.mon.c.trial110.stderr:2026-02-24T18:43:01.263+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 45169) UID: 0 2026-02-24T18:43:01.269 INFO:tasks.ceph.mon.c.trial110.stderr:2026-02-24T18:43:01.263+0000 aca6640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2026-02-24T18:43:04.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2026-02-24T18:43:04.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2026-02-24T18:43:07.363 INFO:tasks.ceph.mon.c:Stopped 2026-02-24T18:43:07.363 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-24T18:43:07.364 DEBUG:teuthology.orchestra.run.trial018:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:43:07.393 INFO:teuthology.orchestra.run.trial018.stdout:2026-02-24T18:35:26.767634+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-24T18:43:07.394 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2026-02-24T18:43:07.395 DEBUG:teuthology.orchestra.run.trial018:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:43:07.462 DEBUG:teuthology.orchestra.run.trial018:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:43:07.528 DEBUG:teuthology.orchestra.run.trial018:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:43:07.594 INFO:teuthology.orchestra.run.trial018.stdout:2026-02-24T18:35:26.767634+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-24T18:43:07.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:43:07.595 DEBUG:teuthology.orchestra.run.trial018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-24T18:43:07.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:43:07.665 DEBUG:teuthology.orchestra.run.trial018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-24T18:43:07.733 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:43:07.734 DEBUG:teuthology.orchestra.run.trial018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2026-02-24T18:43:07.804 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:43:07.804 DEBUG:teuthology.orchestra.run.trial018:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2026-02-24T18:43:07.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:43:07.879 DEBUG:teuthology.orchestra.run.trial032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-24T18:43:07.912 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:43:07.912 DEBUG:teuthology.orchestra.run.trial032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-24T18:43:07.986 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:43:07.986 DEBUG:teuthology.orchestra.run.trial032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2026-02-24T18:43:08.053 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:43:08.053 DEBUG:teuthology.orchestra.run.trial032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2026-02-24T18:43:08.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:43:08.121 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-24T18:43:08.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:43:08.161 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2026-02-24T18:43:08.232 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:43:08.232 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2026-02-24T18:43:08.302 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:43:08.302 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2026-02-24T18:43:08.373 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:43:08.373 DEBUG:teuthology.orchestra.run.trial112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-24T18:43:08.412 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:43:08.412 DEBUG:teuthology.orchestra.run.trial112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2026-02-24T18:43:08.481 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:43:08.481 DEBUG:teuthology.orchestra.run.trial112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2026-02-24T18:43:08.549 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:43:08.550 DEBUG:teuthology.orchestra.run.trial112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2026-02-24T18:43:08.621 INFO:tasks.ceph:Archiving mon data... 2026-02-24T18:43:08.623 DEBUG:teuthology.misc:Transferring archived files from trial018:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/data/mon.a.tgz 2026-02-24T18:43:08.624 DEBUG:teuthology.orchestra.run.trial018:> mktemp 2026-02-24T18:43:08.642 INFO:teuthology.orchestra.run.trial018.stdout:/tmp/tmp.QudTC9VYUw 2026-02-24T18:43:08.643 DEBUG:teuthology.orchestra.run.trial018:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QudTC9VYUw 2026-02-24T18:43:08.751 DEBUG:teuthology.orchestra.run.trial018:> sudo chmod 0666 /tmp/tmp.QudTC9VYUw 2026-02-24T18:43:08.833 DEBUG:teuthology.orchestra.remote:trial018:/tmp/tmp.QudTC9VYUw is 403KB 2026-02-24T18:43:08.931 DEBUG:teuthology.orchestra.run.trial018:> rm -fr /tmp/tmp.QudTC9VYUw 2026-02-24T18:43:08.948 DEBUG:teuthology.misc:Transferring archived files from trial032:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/data/mon.b.tgz 2026-02-24T18:43:08.948 DEBUG:teuthology.orchestra.run.trial032:> mktemp 2026-02-24T18:43:08.967 INFO:teuthology.orchestra.run.trial032.stdout:/tmp/tmp.ONuGzhZ4nN 2026-02-24T18:43:08.967 DEBUG:teuthology.orchestra.run.trial032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ONuGzhZ4nN 2026-02-24T18:43:09.078 DEBUG:teuthology.orchestra.run.trial032:> sudo chmod 0666 /tmp/tmp.ONuGzhZ4nN 2026-02-24T18:43:09.160 DEBUG:teuthology.orchestra.remote:trial032:/tmp/tmp.ONuGzhZ4nN is 403KB 2026-02-24T18:43:09.256 DEBUG:teuthology.orchestra.run.trial032:> rm -fr /tmp/tmp.ONuGzhZ4nN 2026-02-24T18:43:09.273 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/data/mon.c.tgz 2026-02-24T18:43:09.273 DEBUG:teuthology.orchestra.run.trial110:> mktemp 2026-02-24T18:43:09.291 INFO:teuthology.orchestra.run.trial110.stdout:/tmp/tmp.NL8HywI9kZ 2026-02-24T18:43:09.292 DEBUG:teuthology.orchestra.run.trial110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NL8HywI9kZ 2026-02-24T18:43:09.400 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 0666 /tmp/tmp.NL8HywI9kZ 2026-02-24T18:43:09.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2026-02-24T18:43:09.482 DEBUG:teuthology.orchestra.remote:trial110:/tmp/tmp.NL8HywI9kZ is 403KB 2026-02-24T18:43:09.577 DEBUG:teuthology.orchestra.run.trial110:> rm -fr /tmp/tmp.NL8HywI9kZ 2026-02-24T18:43:09.594 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-24T18:43:09.594 DEBUG:teuthology.orchestra.run.trial018:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T18:43:09.597 DEBUG:teuthology.orchestra.run.trial032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T18:43:09.600 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T18:43:09.636 DEBUG:teuthology.orchestra.run.trial112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T18:43:09.930 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-24T18:43:10.679 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-24T18:43:10.679 INFO:tasks.ceph:Archiving crash dumps... 2026-02-24T18:43:10.682 DEBUG:teuthology.misc:Transferring archived files from trial018:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial018/crash 2026-02-24T18:43:10.683 DEBUG:teuthology.orchestra.run.trial018:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T18:43:10.710 DEBUG:teuthology.misc:Transferring archived files from trial032:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial032/crash 2026-02-24T18:43:10.711 DEBUG:teuthology.orchestra.run.trial032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T18:43:10.737 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial110/crash 2026-02-24T18:43:10.739 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T18:43:10.766 DEBUG:teuthology.misc:Transferring archived files from trial112:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial112/crash 2026-02-24T18:43:10.767 DEBUG:teuthology.orchestra.run.trial112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T18:43:10.794 INFO:tasks.ceph:Compressing logs... 2026-02-24T18:43:10.794 DEBUG:teuthology.orchestra.run.trial018:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:10.797 DEBUG:teuthology.orchestra.run.trial032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:10.800 DEBUG:teuthology.orchestra.run.trial110:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:10.811 DEBUG:teuthology.orchestra.run.trial112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:10.819 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57195.log 2026-02-24T18:43:10.832 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.13.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/valgrind/osd.13.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.9.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.13.log.gz 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2026-02-24T18:43:10.833 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/valgrind/osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.9.log.gz 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr: 94.1% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph-osd.1.log: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2026-02-24T18:43:10.834 INFO:teuthology.orchestra.run.trial032.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.9.log: 87.7% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.14.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/valgrind/osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.10.log 2026-02-24T18:43:10.842 INFO:teuthology.orchestra.run.trial110.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/valgrind/osd.14.log: 68.2% -- replaced with /var/log/ceph/valgrind/osd.14.log.gz 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/valgrind/osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.10.log.gz 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr: 94.1% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T18:43:10.843 INFO:teuthology.orchestra.run.trial110.stderr: 87.7% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-24T18:43:10.870 INFO:teuthology.orchestra.run.trial032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-24T18:43:10.871 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45025.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44804.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45025.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45025.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44652.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44804.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45260.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45119.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44652.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45307.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45307.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45260.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45307.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-osd.7.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz -5 --verbose 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: -- /var/log/ceph/valgrind/osd.7.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45166.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.11.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/valgrind/osd.7.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.15.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/valgrind/osd.11.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.11.log.gz 2026-02-24T18:43:10.872 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/valgrind/osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/valgrind/osd.3.log: 68.1% -- replaced with /var/log/ceph/valgrind/osd.15.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44757.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44978.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44978.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44757.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44927.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44710.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44978.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.44927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44927.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45213.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45072.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45072.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45213.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-client.admin.45354.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45072.log.gz 2026-02-24T18:43:10.873 INFO:teuthology.orchestra.run.trial112.stderr:/var/log/ceph/ceph-osd.15.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54239.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55970.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57195.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51159.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59842.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54239.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55970.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54681.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59842.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51159.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51886.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59842.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-mon.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54681.log.gz 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2026-02-24T18:43:10.880 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58961.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51886.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53184.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58079.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45803.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58961.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58079.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53184.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58030.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.45803.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58079.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59500.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45803.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58030.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58030.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-osd.0.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59500.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55872.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59010.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59010.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55872.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57293.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59010.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57293.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2026-02-24T18:43:10.881 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59745.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57293.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58814.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52196.log: /var/log/ceph/ceph-client.admin.59745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57393.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58814.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59745.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58814.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49792.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54190.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57393.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55823.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49792.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54583.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54190.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55823.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49405.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54583.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55823.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59353.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49405.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54583.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51787.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49405.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51664.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59353.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51787.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57442.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51664.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52326.log: 94.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-24T18:43:10.882 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57442.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52742.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55480.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52907.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52742.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55480.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46950.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52907.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57981.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.46950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51694.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51144.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46950.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51694.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57981.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54436.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51694.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59255.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51144.log: /var/log/ceph/ceph-client.admin.54436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56362.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51144.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59255.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54436.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51963.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59255.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59059.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53896.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56362.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59059.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51963.log.gz 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54891.log 2026-02-24T18:43:10.883 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53896.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59059.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56950.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:gzip/var/log/ceph/ceph-client.admin.54891.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49699.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53896.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54141.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54891.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56950.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49971.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54141.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57146.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54141.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51591.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53798.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57146.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51591.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53405.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54387.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53405.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.53798.log.gz --verbose 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: -- /var/log/ceph/ceph-client.admin.58373.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53405.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55431.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54387.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46814.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58373.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.46814.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.56264.log -- replaced with /var/log/ceph/ceph-client.admin.55431.log.gz 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57834.log 2026-02-24T18:43:10.884 INFO:teuthology.orchestra.run.trial018.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46814.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56264.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55382.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57834.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58226.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- 88.3% 92.4% /var/log/ceph/ceph-client.admin.54632.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55382.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54092.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56754.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58226.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54092.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55676.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54092.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58275.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55676.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56754.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55676.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.12.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.8.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58275.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/valgrind/osd.12.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.12.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/valgrind/osd.8.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.8.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57048.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/valgrind/osd.4.log: 68.2% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52812.log 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57048.log: 94.1% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2026-02-24T18:43:10.885 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59598.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52812.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57048.log.gz -5 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55284.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52812.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51113.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59598.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54043.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59304.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55284.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50401.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51113.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54043.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52080.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59304.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55921.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55333.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50401.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52080.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55921.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52715.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56607.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55333.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49869.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56607.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52715.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52197.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51119.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56607.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52197.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49869.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58863.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52197.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56166.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56558.log 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51119.log.gz 2026-02-24T18:43:10.886 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56166.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58863.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58422.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56558.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56166.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58765.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56558.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54940.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58569.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58422.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54940.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58765.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53356.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54940.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57932.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45970.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58569.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53356.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.45970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57932.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57687.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45970.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45734.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57687.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.45734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55186.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57687.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45734.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51063.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56999.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55186.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51063.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51063.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53994.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56999.log.gz 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56068.log 2026-02-24T18:43:10.887 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53994.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56411.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51665.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56068.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50389.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56411.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56705.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51665.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56460.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50389.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49359.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59402.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56460.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56705.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53154.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56460.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59402.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49359.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50396.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59402.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57244.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr: 87.5%/var/log/ceph/ceph-client.admin.50396.log: -- replaced with /var/log/ceph/ceph-client.admin.53154.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58667.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57244.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50396.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54338.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58667.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57244.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49497.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58667.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54338.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49497.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56509.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph.audit.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58324.log 2026-02-24T18:43:10.888 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58324.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56509.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54534.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52195.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58324.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54534.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51112.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55578.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54534.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52081.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51112.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39053.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52081.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51112.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51072.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr: 87.3%/var/log/ceph/ceph.tmp-client.admin.39053.log: -- replaced with /var/log/ceph/ceph-client.admin.52081.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39053.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56656.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56656.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51072.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55088.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56656.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59206.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50802.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59206.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55088.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56215.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59206.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59157.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50802.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59157.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56215.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57097.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51109.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59157.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55774.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53847.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55774.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57097.log.gz 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55627.log 2026-02-24T18:43:10.889 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53847.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55774.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51074.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55627.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53847.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51122.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55627.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53504.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51074.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57343.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51122.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58177.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57343.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53504.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59696.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57343.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58471.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59696.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58177.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59451.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59696.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59108.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55235.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58471.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59108.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59451.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55235.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59108.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50419.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55235.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53230.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53062.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52582.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53230.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57540.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52582.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53230.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50424.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57540.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52582.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55039.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50424.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57540.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50424.log.gz 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58716.log 2026-02-24T18:43:10.890 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55039.log: /var/log/ceph/ceph-client.admin.50402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54989.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55039.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58716.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53299.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54989.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58716.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58618.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53299.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54989.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56313.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58618.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53299.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52467.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56313.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58618.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56803.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52467.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56313.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52467.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54829.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56803.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.45757.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54731.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54829.log: gzip -5 --verbose 88.7% -- /var/log/ceph/ceph-client.admin.54289.log -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53004.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54829.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57883.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54731.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54289.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53004.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56852.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57883.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59795.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56852.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59795.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52363.log 2026-02-24T18:43:10.891 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53455.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59795.log.gzgzip 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51411.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57589.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52363.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51411.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51411.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55725.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58128.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57589.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.55725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58912.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55725.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58128.log: 87.7%gzip -- replaced with /var/log/ceph/ceph.log.gz -5 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58912.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58128.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57491.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52849.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58912.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57491.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56019.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53700.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57491.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53700.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56019.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59647.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50421.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53700.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57638.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52934.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49313.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57638.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59647.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56901.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50425.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52546.log 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52934.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50425.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56901.log.gz 2026-02-24T18:43:10.892 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54485.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.52546.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50425.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.54485.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52546.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53553.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53651.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54485.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57736.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.53553.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51237.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.57736.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53553.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.51237.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57736.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56117.log 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial032.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2026-02-24T18:43:10.893 INFO:teuthology.orchestra.run.trial032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51237.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59549.log 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.56117.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49769.log 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.59549.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56117.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50393.log 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.49769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58520.log 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59549.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.50393.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49769.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr:/var/log/ceph/ceph-client.admin.58520.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50393.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial018.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58520.log.gz 2026-02-24T18:43:10.894 INFO:teuthology.orchestra.run.trial110.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-24T18:43:10.896 INFO:teuthology.orchestra.run.trial110.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2026-02-24T18:43:10.897 INFO:teuthology.orchestra.run.trial110.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-24T18:43:10.898 INFO:teuthology.orchestra.run.trial110.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2026-02-24T18:43:10.908 INFO:teuthology.orchestra.run.trial018.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2026-02-24T18:43:10.912 INFO:teuthology.orchestra.run.trial112.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-24T18:43:10.913 INFO:teuthology.orchestra.run.trial112.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-24T18:43:10.914 INFO:teuthology.orchestra.run.trial018.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-24T18:43:10.917 INFO:teuthology.orchestra.run.trial110.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-24T18:43:10.919 INFO:teuthology.orchestra.run.trial110.stderr: 2026-02-24T18:43:10.919 INFO:teuthology.orchestra.run.trial110.stderr:real 0m0.096s 2026-02-24T18:43:10.919 INFO:teuthology.orchestra.run.trial110.stderr:user 0m0.334s 2026-02-24T18:43:10.919 INFO:teuthology.orchestra.run.trial110.stderr:sys 0m0.030s 2026-02-24T18:43:10.923 INFO:teuthology.orchestra.run.trial112.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-24T18:43:10.929 INFO:teuthology.orchestra.run.trial112.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2026-02-24T18:43:10.929 INFO:teuthology.orchestra.run.trial018.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-24T18:43:10.930 INFO:teuthology.orchestra.run.trial112.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2026-02-24T18:43:10.931 INFO:teuthology.orchestra.run.trial112.stderr: 2026-02-24T18:43:10.932 INFO:teuthology.orchestra.run.trial112.stderr:real 0m0.076s 2026-02-24T18:43:10.932 INFO:teuthology.orchestra.run.trial112.stderr:user 0m0.260s 2026-02-24T18:43:10.932 INFO:teuthology.orchestra.run.trial112.stderr:sys 0m0.046s 2026-02-24T18:43:10.933 INFO:teuthology.orchestra.run.trial032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-24T18:43:10.934 INFO:teuthology.orchestra.run.trial032.stderr: 2026-02-24T18:43:10.934 INFO:teuthology.orchestra.run.trial032.stderr:real 0m0.122s 2026-02-24T18:43:10.934 INFO:teuthology.orchestra.run.trial032.stderr:user 0m0.314s 2026-02-24T18:43:10.934 INFO:teuthology.orchestra.run.trial032.stderr:sys 0m0.034s 2026-02-24T18:43:11.006 INFO:teuthology.orchestra.run.trial018.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-24T18:43:11.008 INFO:teuthology.orchestra.run.trial018.stderr: 2026-02-24T18:43:11.008 INFO:teuthology.orchestra.run.trial018.stderr:real 0m0.197s 2026-02-24T18:43:11.008 INFO:teuthology.orchestra.run.trial018.stderr:user 0m0.484s 2026-02-24T18:43:11.008 INFO:teuthology.orchestra.run.trial018.stderr:sys 0m0.135s 2026-02-24T18:43:11.009 INFO:tasks.ceph:Archiving logs... 2026-02-24T18:43:11.010 DEBUG:teuthology.misc:Transferring archived files from trial018:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial018/log 2026-02-24T18:43:11.011 DEBUG:teuthology.orchestra.run.trial018:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T18:43:11.381 DEBUG:teuthology.misc:Transferring archived files from trial032:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial032/log 2026-02-24T18:43:11.382 DEBUG:teuthology.orchestra.run.trial032:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T18:43:11.498 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial110/log 2026-02-24T18:43:11.499 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T18:43:11.590 DEBUG:teuthology.misc:Transferring archived files from trial112:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial112/log 2026-02-24T18:43:11.592 DEBUG:teuthology.orchestra.run.trial112:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T18:43:11.681 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1919, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 56, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 48, in nested if exit(*exc): ^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-24T18:43:11.683 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-24T18:43:11.712 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T18:43:11.726 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T18:43:11.726 DEBUG:teuthology.orchestra.run.trial018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:43:11.729 DEBUG:teuthology.orchestra.run.trial032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:43:11.731 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:43:11.734 DEBUG:teuthology.orchestra.run.trial112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:43:12.023 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T18:43:12.034 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T18:43:12.034 DEBUG:teuthology.orchestra.run.trial018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:43:12.035 DEBUG:teuthology.orchestra.run.trial032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:43:12.037 DEBUG:teuthology.orchestra.run.trial110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:43:12.039 DEBUG:teuthology.orchestra.run.trial112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:43:12.049 INFO:teuthology.orchestra.run.trial018.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:43:12.051 INFO:teuthology.orchestra.run.trial032.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:43:12.053 INFO:teuthology.orchestra.run.trial110.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:43:12.055 INFO:teuthology.orchestra.run.trial112.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:43:12.078 INFO:teuthology.orchestra.run.trial018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:43:12.078 INFO:teuthology.orchestra.run.trial018.stdout:=============================================================================== 2026-02-24T18:43:12.079 INFO:teuthology.orchestra.run.trial018.stdout:^* vps5.drown.org 2 7 377 67 -205us[ -242us] +/- 22ms 2026-02-24T18:43:12.079 INFO:teuthology.orchestra.run.trial018.stdout:^- linode1.ernest-doss.org 3 7 377 67 -1089us[-1089us] +/- 98ms 2026-02-24T18:43:12.079 INFO:teuthology.orchestra.run.trial018.stdout:^+ 198.71.50.75 3 7 377 6 +304us[ +304us] +/- 36ms 2026-02-24T18:43:12.079 INFO:teuthology.orchestra.run.trial018.stdout:^+ us-east-2.clearnet.pw 2 7 377 1 -356us[ -356us] +/- 48ms 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:=============================================================================== 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:^- jane.qotw.net 2 7 377 8 +1791us[+1881us] +/- 25ms 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:^- vps6.drown.org 2 7 377 5 +2329us[+2420us] +/- 23ms 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:^+ 45.55.126.202 2 7 377 70 -526us[ -415us] +/- 9776us 2026-02-24T18:43:12.083 INFO:teuthology.orchestra.run.trial110.stdout:^* time.cloudflare.com 3 6 377 2 +412us[ +504us] +/- 6506us 2026-02-24T18:43:12.087 INFO:teuthology.orchestra.run.trial112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial112.stdout:=============================================================================== 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial112.stdout:^- 66.85.78.80 2 7 377 65 -901us[-1226us] +/- 48ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial112.stdout:^+ 50.218.103.254 2 7 304 456 +372us[ +60us] +/- 22ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial112.stdout:^- 172-233-157-223.ip.linod> 2 7 377 2 -10ms[ -10ms] +/- 123ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial112.stdout:^* ntp4.glypnod.com 2 7 377 4 -821us[-1148us] +/- 12ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:=============================================================================== 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:^+ 172-104-209-204.ip.linod> 4 6 377 5 +3153us[+3153us] +/- 31ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:^* chi3.us.ntp.li 2 6 377 5 -1119us[-1115us] +/- 11ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:^- arm1.maxhost.io 2 7 377 70 -1184us[-1132us] +/- 54ms 2026-02-24T18:43:12.088 INFO:teuthology.orchestra.run.trial032.stdout:^- us-west-1.clearnet.pw 2 7 377 70 +2491us[+2544us] +/- 58ms 2026-02-24T18:43:12.089 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T18:43:12.100 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T18:43:12.101 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T18:43:12.115 DEBUG:teuthology.orchestra.run.trial018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T18:43:12.175 DEBUG:teuthology.orchestra.run.trial032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T18:43:12.231 DEBUG:teuthology.orchestra.run.trial110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T18:43:12.285 DEBUG:teuthology.orchestra.run.trial112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T18:43:12.340 DEBUG:teuthology.orchestra.run.trial018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:43:12.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:12.369 DEBUG:teuthology.orchestra.run.trial032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:43:12.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:12.397 DEBUG:teuthology.orchestra.run.trial110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:43:12.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:12.426 DEBUG:teuthology.orchestra.run.trial112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:43:12.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:12.454 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T18:43:12.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T18:43:12.484 INFO:teuthology.task.internal:Duration was 1603.540877 seconds 2026-02-24T18:43:12.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T18:43:12.499 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T18:43:12.500 DEBUG:teuthology.orchestra.run.trial018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:43:12.503 DEBUG:teuthology.orchestra.run.trial032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:43:12.505 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:43:12.508 DEBUG:teuthology.orchestra.run.trial112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:43:12.539 INFO:teuthology.orchestra.run.trial018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:43:12.542 INFO:teuthology.orchestra.run.trial032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:43:12.544 INFO:teuthology.orchestra.run.trial110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:43:12.545 INFO:teuthology.orchestra.run.trial112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:43:12.922 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T18:43:12.923 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial018.front.sepia.ceph.com 2026-02-24T18:43:12.923 DEBUG:teuthology.orchestra.run.trial018:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:43:12.944 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial032.front.sepia.ceph.com 2026-02-24T18:43:12.945 DEBUG:teuthology.orchestra.run.trial032:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:43:12.980 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial110.front.sepia.ceph.com 2026-02-24T18:43:12.980 DEBUG:teuthology.orchestra.run.trial110:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:43:13.001 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial112.front.sepia.ceph.com 2026-02-24T18:43:13.001 DEBUG:teuthology.orchestra.run.trial112:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:43:13.021 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T18:43:13.022 DEBUG:teuthology.orchestra.run.trial018:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.024 DEBUG:teuthology.orchestra.run.trial032:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.027 DEBUG:teuthology.orchestra.run.trial110:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.044 DEBUG:teuthology.orchestra.run.trial112:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.123 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T18:43:13.123 DEBUG:teuthology.orchestra.run.trial018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:13.126 DEBUG:teuthology.orchestra.run.trial032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:13.128 DEBUG:teuthology.orchestra.run.trial110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:13.131 DEBUG:teuthology.orchestra.run.trial112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:43:13.147 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:43:13.148 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:43:13.148 INFO:teuthology.orchestra.run.trial018.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:43:13.148 INFO:teuthology.orchestra.run.trial018.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.148 INFO:teuthology.orchestra.run.trial018.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:43:13.149 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:43:13.150 INFO:teuthology.orchestra.run.trial032.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:43:13.150 INFO:teuthology.orchestra.run.trial032.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.150 INFO:teuthology.orchestra.run.trial032.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:43:13.150 INFO:teuthology.orchestra.run.trial032.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:43:13.152 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:43:13.153 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:43:13.153 INFO:teuthology.orchestra.run.trial110.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.153 INFO:teuthology.orchestra.run.trial110.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:43:13.153 INFO:teuthology.orchestra.run.trial110.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:43:13.155 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:43:13.155 INFO:teuthology.orchestra.run.trial112.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:43:13.155 INFO:teuthology.orchestra.run.trial112.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:43:13.156 INFO:teuthology.orchestra.run.trial112.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:43:13.156 INFO:teuthology.orchestra.run.trial112.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:43:13.158 INFO:teuthology.orchestra.run.trial018.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:43:13.160 INFO:teuthology.orchestra.run.trial032.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:43:13.163 INFO:teuthology.orchestra.run.trial110.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:43:13.166 INFO:teuthology.orchestra.run.trial112.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:43:13.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T18:43:13.183 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T18:43:13.184 DEBUG:teuthology.orchestra.run.trial018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:43:13.225 DEBUG:teuthology.orchestra.run.trial032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:43:13.249 DEBUG:teuthology.orchestra.run.trial110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:43:13.275 DEBUG:teuthology.orchestra.run.trial112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:43:13.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T18:43:13.320 DEBUG:teuthology.orchestra.run.trial018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.323 DEBUG:teuthology.orchestra.run.trial032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.326 DEBUG:teuthology.orchestra.run.trial110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.329 DEBUG:teuthology.orchestra.run.trial112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.345 INFO:teuthology.orchestra.run.trial018.stdout:kernel.core_pattern = core 2026-02-24T18:43:13.348 INFO:teuthology.orchestra.run.trial032.stdout:kernel.core_pattern = core 2026-02-24T18:43:13.350 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern = core 2026-02-24T18:43:13.365 INFO:teuthology.orchestra.run.trial112.stdout:kernel.core_pattern = core 2026-02-24T18:43:13.379 DEBUG:teuthology.orchestra.run.trial018:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:13.416 DEBUG:teuthology.orchestra.run.trial032:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:13.432 DEBUG:teuthology.orchestra.run.trial110:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:13.450 DEBUG:teuthology.orchestra.run.trial112:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:43:13.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:43:13.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T18:43:13.484 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T18:43:13.485 DEBUG:teuthology.misc:Transferring archived files from trial018:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial018 2026-02-24T18:43:13.485 DEBUG:teuthology.orchestra.run.trial018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T18:43:13.523 DEBUG:teuthology.misc:Transferring archived files from trial032:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial032 2026-02-24T18:43:13.523 DEBUG:teuthology.orchestra.run.trial032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T18:43:13.561 DEBUG:teuthology.misc:Transferring archived files from trial110:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial110 2026-02-24T18:43:13.561 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T18:43:13.597 DEBUG:teuthology.misc:Transferring archived files from trial112:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68651/remote/trial112 2026-02-24T18:43:13.598 DEBUG:teuthology.orchestra.run.trial112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T18:43:13.633 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T18:43:13.633 DEBUG:teuthology.orchestra.run.trial018:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T18:43:13.636 DEBUG:teuthology.orchestra.run.trial032:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T18:43:13.639 DEBUG:teuthology.orchestra.run.trial110:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T18:43:13.642 DEBUG:teuthology.orchestra.run.trial112:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T18:43:13.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T18:43:13.701 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T18:43:13.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T18:43:13.718 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T18:43:13.718 DEBUG:teuthology.orchestra.run.trial018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T18:43:13.721 DEBUG:teuthology.orchestra.run.trial032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T18:43:13.724 DEBUG:teuthology.orchestra.run.trial110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T18:43:13.727 DEBUG:teuthology.orchestra.run.trial112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T18:43:13.735 INFO:teuthology.orchestra.run.trial018.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 18:43 /home/ubuntu/cephtest 2026-02-24T18:43:13.738 INFO:teuthology.orchestra.run.trial032.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 18:43 /home/ubuntu/cephtest 2026-02-24T18:43:13.741 INFO:teuthology.orchestra.run.trial110.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 18:43 /home/ubuntu/cephtest 2026-02-24T18:43:13.743 INFO:teuthology.orchestra.run.trial112.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 18:43 /home/ubuntu/cephtest 2026-02-24T18:43:13.744 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T18:43:13.761 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T18:43:13.799 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} duration: 1603.5408771038055 failure_reason: '"2026-02-24T18:35:26.767634+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-24T18:43:13.799 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T18:43:13.899 INFO:teuthology.run:FAIL